Python System Monitoring and Maintenance

When developing IoT devices that require system data monitoring, or performing system maintenance tasks on Linux and other operating systems, Python scripts are often used to simplify routine operations. This is particularly crucial in operational maintenance and development, where monitoring various hardware metrics of the system and detecting anomalies is of paramount importance. In Linux, there are many system commands, such as ps, top, free, etc., that allow for constant monitoring of the system's running state. To retrieve this information, Python can execute these commands using the subprocess module and parse the results. However, this approach can be cumbersome, especially when a lot of parsing code is needed.

In Python, there's an easier way to access system operational data: the ``psutil` module. Psutil stands for “process and system utilities”. It's a cross-platform library supporting Linux/UNIX/OSX/Windows and is an indispensable module for IoT development and operational work, enabling system monitoring with minimal code.

Installing psutil

To install psutil on various operating systems, use the following command:

$ pip install psutil

(If you encounter a Permission Denied error, please retry with sudo.)

Accessing Various System Information with Psutil

Psutil allows us to easily access various system information. Here are some examples of its common features:

1. Retrieving CPU Information

Using psutil to monitor CPU usage, core count, and other vital metrics is helpful.

import psutil

# Retrieve complete CPU information
print(psutil.cpu_times())

# Get the number of logical CPUs
print(psutil.cpu_count())

# Monitor CPU usage over three seconds
# The interval=1 parameter indicates once per second
for i in range(3):
    print(psutil.cpu_percent(interval=1))

2. Retrieving Memory Information

Monitoring memory is crucial for ensuring efficient program execution.

import psutil

# Retrieve complete memory information
print(psutil.virtual_memory())

# Get available memory specifically
print(psutil.virtual_memory().available)

# Retrieve swap memory information
print(psutil.swap_memory())

3. Retrieving Disk Information

Monitoring disk information helps prevent issues with insufficient storage space.

import psutil

# Retrieve disk partition information
print(psutil.disk_partitions())

# Monitor disk usage
print(psutil.disk_usage('/'))

# Retrieve disk I/O
print(psutil.disk_io_counters())

4. Retrieving Network Information

Monitoring network information is useful for understanding data transmission and network status.

import psutil

# Retrieve total network I/O information
print(psutil.net_io_counters())

# Get information for each network interface
print(psutil.net_if_addrs())

# Monitor network interface status
print(psutil.net_if_stats())

5. Retrieving System User Information

Understanding who is currently logged into the system.

import psutil

# Retrieve user login information
print(psutil.users())

Other Uses

Psutil is not limited to retrieving system information; it can also handle more complex system management tasks. Here are some detailed introductions and code examples of its advanced uses:

Process Management

Psutil can list and manage processes within the system. It allows us to retrieve process information, terminate processes, or create new ones.

import psutil

# List all process IDs
for proc in psutil.process_iter(['pid', 'name']):
    print(proc.info)

# Terminate a specific process (by process ID)
pid = 12345  # An assumed process ID
if psutil.pid_exists(pid):
    p = psutil.Process(pid)
    p.terminate()  # or use p.kill()

System Reboot and Shutdown

Psutil can also be used to programmatically control system reboot or shutdown, which is especially useful in remote management of servers or IoT devices.

import psutil

# Reboot the system
psutil.reboot()

# Shutdown the system
psutil.shutdown()

Battery Information and Power Plug Status

For portable devices, monitoring battery status and power plug status is an important feature. Psutil can provide this information.

import psutil

# Retrieve battery status
battery = psutil.sensors_battery()
print(f"Battery remaining: {battery.percent}%")
print(f"Power plugged in: {'Yes' if battery.power_plugged else 'No'}")

Monitoring File System Changes

Psutil can be used to monitor changes in the file system, such as space usage and changes in mount points.

import psutil

# Check disk usage
disk_usage = psutil.disk_usage('/')
print(f"Total space: {disk_usage.total / (1024**3):.2f} GB")
print(f"Used space: {disk_usage.used / (1024**3):.2f} GB")
print(f"Free space: {disk_usage.free / (1024**3):.2f} GB")

Monitoring Network Connections

Psutil can also be used to monitor network connections, including current network connection information.

import psutil

# Retrieve network connection information
connections = psutil.net_connections()
for conn in connections:
    print(conn)

For more usage details and API documentation, please refer to the psutil official website: psutil.readthedocs.io.

Script Examples

Routine Linux Maintenance Monitoring and Alerts

import psutil
import time
import smtplib
from email.mime.text import MIMEText
from email.header import Header
import logging

# Email sending settings
smtp_server = "smtp.example.com"  # SMTP server address
smtp_port = 587  # SMTP server port number
email_user = "your_email@example.com"  # Email username
email_pass = "your_password"  # Email password
receiver_email = "receiver@example.com"  # Receiver's email

# Logging configuration
logging.basicConfig(filename='system_monitor.log', level=logging.INFO,
                    format='%(asctime)s:%(levelname)s:%(message)s')

def send_email(subject, body):
    """ Send an email """
    message = MIMEText(body, 'plain', 'utf-8')
    message['From'] = Header("System Monitor", 'utf-8')
    message['To'] = Header("Admin", 'utf-8')
    message['Subject'] = Header(subject, 'utf-8')

    try:
        smtp = smtplib.SMTP(smtp_server, smtp_port)
        smtp.starttls()  # Enable secure transport mode
        smtp.login(email_user, email_pass)
        smtp.sendmail(email_user, receiver_email, message.as_string())
        smtp.quit()
        logging.info("Email sent successfully")
    except smtplib.SMTPException as e:
        logging.error("Email sending failed: " + str(e))

def check_system():
    """ Check system status and send warnings if necessary """
    if check_cpu_usage():
        warning = "Warning: High CPU usage!"
        logging.warning(warning)
        send_email("CPU Warning", warning)

    if check_memory_usage():
        warning = "Warning: High memory usage!"
        logging.warning(warning)
        send_email("Memory Warning", warning)

    if check_disk_usage():
        warning = "Warning: High disk usage!"
        logging.warning(warning)
        send_email("Disk Warning", warning)

def main():
    print("Starting system monitoring...")
    while True:
        check_system()
        network_data = check_network_activity()
        print(f"Current network activity: {network_data} bytes")
        time.sleep(5)

if __name__ == "__main__":
    main()

2.IoT Device System Data Upload

If you're an IoT device developer needing to monitor system operation data for Linux-based IoT devices (like Raspberry Pi), regularly monitoring and uploading CPU and memory usage data is crucial. This script is ideal for scenarios where remote monitoring of hardware performance is needed, such as tracking the health or performance bottlenecks of IoT projects. It helps maintenance teams or developers promptly understand the device's operational status and make data-driven optimization or maintenance decisions.

First, install the necessary libraries:

$ pip install psutil schedule requests

Then, here is the example script:

import psutil
import schedule
import time
import requests
import logging
import json

# Logging configuration
logging.basicConfig(filename='iot_monitor.log', level=logging.INFO,
                    format='%(asctime)s:%(levelname)s:%(message)s')

# API setup (assumed API address and endpoint)
api_url = "http://example.com/api/upload"

def get_system_stats():
    """ Retrieve system's CPU and memory usage information """
    cpu_usage = psutil.cpu_percent()
    memory_usage = psutil.virtual_memory().percent
    return {"cpu": cpu_usage, "memory": memory_usage}

def upload_data():
    """ Upload collected data to the server """
    data = get_system_stats()
    try:
        response = requests.post(api_url, json=data)
        if response.status_code == 200:
            logging.info("Data uploaded successfully")
        else:
            logging.error(f"Data upload failed: {response.status_code}")
    except requests.exceptions.RequestException as e:
        logging.error(f"Request exception: {e}")

def job():
    """ Task to be executed periodically """
    logging.info("Starting data collection")
    upload_data()

def main():
    # Define a scheduled task: to be executed every 10 minutes
    schedule.every(10).minutes.do(job)

    while True:
        schedule.run_pending()
        time.sleep(1)

if __name__ == "__main__":
    main()