Disk I/O Monitoring: `iostat`, `iotop`

Disk Input/Output (I/O) refers to the data moving between your computer's CPU and its storage devices (like hard drives or SSDs).

Monitoring disk I/O is crucial for understanding system performance. Slow disk I/O can be a major bottleneck, making your applications and overall system feel sluggish, even if your CPU and memory are underutilized.

The iostat command is part of the sysstat package and is used to report CPU statistics and device I/O statistics.

It provides a system-wide view of disk activity, helping you identify if your storage devices are struggling to keep up with demand.

Running iostat without any options gives you a snapshot of current I/O performance.

A typical iostat output shows several key metrics:

• Device: The name of the storage device (e.g., sda, nvme0n1).
• tps: Transfers per second (read + write).
• kB_read/s: Kilobytes read from the device per second.
• kB_wrtn/s: Kilobytes written to the device per second.
• kB_read: Total kilobytes read since boot.
• kB_wrtn: Total kilobytes written since boot.

These give you a quick overview of the overall data flow.

For deeper insights, use iostat -x to see extended statistics. Pay attention to:

• r/s, w/s: Reads/writes per second (similar to tps, but separated).
• rkB/s, wkB/s: Read/write kilobytes per second.
• %util: Percentage of time the device was busy. High %util (near 100%) indicates a potential bottleneck.
• await: The average time (in milliseconds) for I/O requests issued to the device to be served. This includes time spent in queue and service time.

High await values suggest I/O requests are waiting too long.

To observe I/O activity over time, you can specify an interval in seconds. This allows you to see how stats change during different workloads.

For example, iostat -d 2 3 will display disk statistics every 2 seconds, for a total of 3 reports. The -d flag limits the output to only disk statistics.

If you have multiple storage devices, you might want to monitor a specific one. You can append the device name to the iostat command.

For instance, to monitor the sda device with extended statistics every 5 seconds, you would use iostat -x 5 sda.

This helps pinpoint issues on individual drives.

While iostat provides system-wide disk I/O statistics, iotop gives you a more granular, process-level view.

iotop shows which processes are actively performing disk I/O, similar to how top shows CPU usage per process. This is invaluable for identifying specific applications or services hogging disk resources.

When you run iotop, you'll see a real-time list of processes and their I/O activity. Key columns include:

• TID/PID: Thread or Process ID.
• USER: The user owning the process.
• DISK READ: Current disk read rate of the process.
• DISK WRITE: Current disk write rate of the process.
• SWAPIN: Percentage of time spent swapping.
• IO%: Total I/O percentage for the process.

By default, it sorts by IO%, making it easy to spot I/O-intensive tasks.

iotop is interactive by default, but you can use flags for specific views. For example, iotop -oPa shows only processes with active I/O, aggregated for each process, and non-interactive.

This helps quickly identify which processes are causing high disk activity and potentially impacting overall system performance.

Which iostat metric directly indicates the percentage of time a storage device is busy processing requests?

In this lesson, you've learned how to use two powerful Linux commands for disk I/O monitoring:

• iostat: Provides system-wide disk I/O statistics, including transfers per second, read/write rates, and crucial metrics like %util (device utilization) and await (request latency). It's great for identifying overall disk bottlenecks.
• iotop: Offers a real-time, process-level view of disk I/O, helping you pinpoint specific applications or services that are consuming significant disk resources.

Mastering these tools is essential for diagnosing and resolving performance issues related to storage.