How to Track Down/Find Cause of Linux high disk iowait/wa -

How to Track Down/Find Cause of Linux high disk iowait/wa

top - 09:34:12 up 2 days, 20:57,  2 users,  load average: 1.83, 1.99, 2.03
Tasks:  59 total,   2 running,  57 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.0%sy,  0.0%ni,  0.0%id, 99.7%wa,  0.0%hi,  0.0%si,  0.0%st

That 99.7% wa is iowait, it means the server is waiting for a process to complete an IO operation or in plain English, there is a delay in reading and/or writing data to the hard drive.

Here are some ways to get an idea of what is going on (in RPM based Distros this package is called "sysstat") and contains the very helpful binary called "iostat".  There's another package I like called "dstat" that does a great job too.

Run iostat -d 5 (it refreshes the data every 5 seconds):

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda1              2.07         5.27        28.66    1309570    7120384
sda2              0.00         0.03         0.04       7496       8832

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda1             27.15         0.00       357.68          0       1792
sda2              0.00         0.00         0.00          0          0

What we're seeing is that the iowait must be caused primarily by blocks being written to /dev/sda1 which is mounted as / (my root filesystem).  So now we know the main cause is by an application constantly writing to the root partition, but we still don't know what process is directly responsible for this.

If you have a newer kernel it's often possible to view processes by io usage, you can test for this by running: cat /proc/self/io

If the file is not found then you don't have io stats enabled in your kernel.  Then you can check a suspected process by running cat /proc/13/io (replace the 13 with the PID you want to check).

read_bytes: 0
write_bytes: 0
cancelled_write_bytes: 0

 


  • css responsive images
  • responsive table without changing much code solution
  • yum how to install old obsolete packages
  • PHP Howto Store Value of Included File Output Into Variable
  • PHP Migration from 5.3 to 5.4+ and dealing with deprecated functions
  • ffmpeg vidstab to stabilize video
  • userdel user userdel: cannot lock /etc/passwd; try again later.
  • mdadm how to mount inactive array
  • How to find and mount mdadm arrays automatically
  • M2Crypto.SSL.Checker.WrongHost: Peer certificate subjectAltName does not match host, expected fedora-archive.ip-connect.vn.ua, got DNS:mirror.ip-connect.vn.ua
  • [Wed Sep 20 15:34:44 2017] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Wed Sep 20 15:34:44 2017] [error] Init: Unable to read server certificate from file /www/ssl-certs/server.crt [Wed Sep 20 15:34:44 2017] [error] SSL Library Err
  • linux how to answer yes to copy
  • linux cp and mv will not overwrite due to alias!
  • ERROR 2006 (HY000) at line 567: MySQL server has gone away
  • vbulletin 4.2.5 after upgrading from 3.6 white screen fatal php errors
  • iptables v1.4.7: can't initialize iptables table `NAT': Table does not exist (do you need to insmod?) Perhaps iptables or your kernel needs to be upgraded.
  • Linux and FreeBSD how to set time and date
  • FreeBSD/OpenBSD OpenVPN Client error "Cannot allocate TUN/TAP dev dynamically"
  • kdenlive - No LADSPA plugins were found! Check your LADSPA_PATH environment variable. [producer_xml] failed to load transition "qtblend"
  • /usr/bin/supermin-helper exited with error status 1. To see full error messages you may need to enable debugging. See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs at /usr/bin/virt-list-partitions line 177.