How to Track Down/Find Cause of Linux high disk iowait/wa -

How to Track Down/Find Cause of Linux high disk iowait/wa

top - 09:34:12 up 2 days, 20:57,  2 users,  load average: 1.83, 1.99, 2.03
Tasks:  59 total,   2 running,  57 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.0%sy,  0.0%ni,  0.0%id, 99.7%wa,  0.0%hi,  0.0%si,  0.0%st

That 99.7% wa is iowait, it means the server is waiting for a process to complete an IO operation or in plain English, there is a delay in reading and/or writing data to the hard drive.

Here are some ways to get an idea of what is going on (in RPM based Distros this package is called "sysstat") and contains the very helpful binary called "iostat".  There's another package I like called "dstat" that does a great job too.

Run iostat -d 5 (it refreshes the data every 5 seconds):

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda1              2.07         5.27        28.66    1309570    7120384
sda2              0.00         0.03         0.04       7496       8832

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda1             27.15         0.00       357.68          0       1792
sda2              0.00         0.00         0.00          0          0

What we're seeing is that the iowait must be caused primarily by blocks being written to /dev/sda1 which is mounted as / (my root filesystem).  So now we know the main cause is by an application constantly writing to the root partition, but we still don't know what process is directly responsible for this.

If you have a newer kernel it's often possible to view processes by io usage, you can test for this by running: cat /proc/self/io

If the file is not found then you don't have io stats enabled in your kernel.  Then you can check a suspected process by running cat /proc/13/io (replace the 13 with the PID you want to check).

read_bytes: 0
write_bytes: 0
cancelled_write_bytes: 0

 


  • named[1525]: error (broken trust chain) resolving 'min-api.cryptocompare.com/A/IN': 173.245.58.78#53 solution
  • Linux Ubuntu Mint Debian cannot play xvid,x264,mp4 or any videos codec issue - xplayer Could not initialize supporting library.
  • wget howto output to terminal or script instead of file
  • imagemagick convert to chop off top of image
  • How To Set or Change Useragent in Firefox and Seamonkey
  • How To Turn Any .iso file into a bootable USB drive
  • Linux Mint Ubuntu Debian How To Disable Webcam Automatically Onboot To Prevent Spying and Privacy Violations
  • IcedTea Java Web Viewer Config Utility How To Enter
  • Debian Linux Mint Ubuntu Disable Automatic apt-get update
  • How To Install Seamonkey Web Browser on Debian Ubuntu Linux Mint
  • Prevent SSH Bruteforce and Hacks By Disabling Password Authentication
  • SMF Forums / Simple Machines Forums Not Displaying Images Theme or Styles Properly using 127.0.0.1
  • solution mysqldump: Got error: 1044: Access denied for user 'user'@'localhost' to database 'thedb' when using LOCK TABLES
  • MySQL How To Grant Access To ALL Databases For Export and Backup Purposes
  • mdadm how to stop or start a check
  • vzquota : (error) Quota on syscall for id 4532: No such file or directory vzquota on failed [3] OpenVZ Error and Solution
  • Apache htaccess Custom ErrorDocument not working properly for root home page 403 Error Issue and Solution
  • syslinux / pxelinux how to boot from local drive how to
  • samba how to listen on specific IP only
  • How To Install Windows Server 7 8 10 12 2008 2012 2014 2016 Servers Desktops using Linux tftp, dhcpd and samba