Here's a proven example of what a bad hard drive can do, it was technically functioning OK in a RAID array but the system became extremely low and the load become high and IOWAIT was even higher and I always thought it was a bad application. The truth is that this failing 1TB Hitachi has slowly gotten worse and caused huge slowdowns, (eg. 100% load on Thunderbird waiting for e-mails to load etc..). After swapping it out, tabs change instantly, emails are not lagged, and Firefox does slow down or lag. Basically generally slowness in your system that can't be explained or corrected is often due to a bad hard drive causing high IOWAIT which results in extreme lag.
Look below at all of the UDMA_CRC_ERRORS
199 UDMA_CRC_Error_Count 0x000a 200 200 000 Old_age Always - 122
And the other error messages from the SMART log:
Error 122 occurred at disk power-on lifetime: 20037 hours (834 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
84 51 00 00 00 00 00
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ec 00 00 00 00 00 a0 08 25d+05:58:47.630 IDENTIFY DEVICE
60 18 00 27 54 fe a0 04 25d+05:58:46.675 READ FPDMA QUEUED
60 18 00 27 54 fe a0 04 25d+05:58:45.623 READ FPDMA QUEUED
60 18 00 27 54 fe a0 ff 25d+05:58:43.621 READ FPDMA QUEUED
60 18 00 27 54 fe a0 ff 25d+05:58:43.611 READ FPDMA QUEUED
sudo smartctl -a /dev/sdc
smartctl 5.40 2010-03-16 r3077 [i686-pc-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
=== START OF INFORMATION SECTION ===
Device Model: Hitachi HDS721010CLA332
Serial Number: JP2940HQ3ZY7KH
Firmware Version: JP4OA3EA
User Capacity: 1,000,204,886,016 bytes
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: 8
ATA Standard is: ATA-8-ACS revision 4
Local Time is: Mon Apr 8 17:59:38 2013 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x84) Offline data collection activity
was suspended by an interrupting command from host.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (10399) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 173) minutes.
SCT capabilities: (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000b 099 099 016 Pre-fail Always - 2
2 Throughput_Performance 0x0005 135 135 054 Pre-fail Offline - 97
3 Spin_Up_Time 0x0007 135 135 024 Pre-fail Always - 238 (Average 321)
4 Start_Stop_Count 0x0012 100 100 000 Old_age Always - 31
5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0
7 Seek_Error_Rate 0x000b 100 100 067 Pre-fail Always - 0
8 Seek_Time_Performance 0x0005 138 138 020 Pre-fail Offline - 31
9 Power_On_Hours 0x0012 098 098 000 Old_age Always - 20049
10 Spin_Retry_Count 0x0013 100 100 060 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 31
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 43
193 Load_Cycle_Count 0x0012 100 100 000 Old_age Always - 43
194 Temperature_Celsius 0x0002 142 142 000 Old_age Always - 42 (Lifetime Min/Max 21/60)
196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x000a 200 200 000 Old_age Always - 122
SMART Error Log Version: 1
ATA Error Count: 122 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 122 occurred at disk power-on lifetime: 20037 hours (834 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
84 51 00 00 00 00 00
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ec 00 00 00 00 00 a0 08 25d+05:58:47.630 IDENTIFY DEVICE
60 18 00 27 54 fe a0 04 25d+05:58:46.675 READ FPDMA QUEUED
60 18 00 27 54 fe a0 04 25d+05:58:45.623 READ FPDMA QUEUED
60 18 00 27 54 fe a0 ff 25d+05:58:43.621 READ FPDMA QUEUED
60 18 00 27 54 fe a0 ff 25d+05:58:43.611 READ FPDMA QUEUED
Error 121 occurred at disk power-on lifetime: 20037 hours (834 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
84 51 09 36 54 fe 05
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 18 00 27 54 fe 40 08 25d+05:58:43.528 READ FPDMA QUEUED
60 10 00 bf 6c fe 40 08 25d+05:58:43.479 READ FPDMA QUEUED
60 20 00 87 3e fd 40 08 25d+05:58:43.427 READ FPDMA QUEUED
60 18 00 ff 1b fe 40 08 25d+05:58:43.372 READ FPDMA QUEUED
60 20 00 67 fd fd 40 08 25d+05:58:43.345 READ FPDMA QUEUED
Error 120 occurred at disk power-on lifetime: 20037 hours (834 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
84 51 01 de e7 fc 05
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 10 00 cf e7 fc 40 08 25d+05:58:13.286 READ FPDMA QUEUED
60 10 00 af 83 fc 40 08 25d+05:58:13.267 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 08 25d+05:58:13.266 SET FEATURES [Reserved for Serial ATA]
27 00 00 00 00 00 e0 08 25d+05:58:13.265 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 08 25d+05:58:13.261 IDENTIFY DEVICE
Error 119 occurred at disk power-on lifetime: 20037 hours (834 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
84 51 01 be 83 fc 05
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 10 00 af 83 fc 40 08 25d+05:58:09.177 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 08 25d+05:58:09.176 SET FEATURES [Reserved for Serial ATA]
27 00 00 00 00 00 e0 08 25d+05:58:09.175 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 08 25d+05:58:09.171 IDENTIFY DEVICE
ef 03 42 00 00 00 a0 08 25d+05:58:09.170 SET FEATURES [Set transfer mode]
Error 118 occurred at disk power-on lifetime: 20037 hours (834 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
84 51 00 00 00 00 00
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ec 00 00 00 00 00 a0 08 25d+05:57:34.494 IDENTIFY DEVICE
60 10 00 af 83 fc a0 04 25d+05:57:33.538 READ FPDMA QUEUED
60 10 00 af 83 fc a0 04 25d+05:57:32.486 READ FPDMA QUEUED
60 10 00 af 83 fc a0 ff 25d+05:57:30.486 READ FPDMA QUEUED
60 10 00 af 83 fc a0 ff 25d+05:57:30.477 READ FPDMA QUEUED
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 207 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
slows, computerhere, functioning, ok, raid, array, iowait, failing, tb, hitachi, slowdowns, eg, thunderbird, mails, etc, swapping, tabs, instantly, emails, lagged, firefox, lag, generally, slowness, corrected, udma_crc_errors, udma_crc_error_count, old_age, disk, active, idle, completion, registers, sc, sn, cl, ch, dh, commands, cr, fr, dc, powered_up_time, feature_name, ec, fe, fpdma, queued, ff, sudo, smartctl, dev, sdc, pc, linux, gnu, copyright, allen, http, smartmontools, sourceforge, hds, cla, jp, hq, zy, kh, firmware, oa, ea, user, capacity, bytes, database, showall, ata, acs, revision, apr, pdt, capability, enabled, overall, assessment, offline, suspended, auto, execution, previous, completed, capabilities, execute, suspend, scan, supported, conveyance, selective, saves, mode, supports, timer, logging, recommended, polling, extended, sct, feature, attributes, vendor, thresholds, attribute_name, thresh, updated, when_failed, raw_value, raw_read_error_rate, throughput_performance, spin_up_time, start_stop_count, reallocated_sector_ct, seek_error_rate, seek_time_performance, power_on_hours, spin_retry_count, power_cycle_count, off_retract_count, load_cycle_count, temperature_celsius, min, reallocated_event_count, current_pending_sector, offline_uncorrectable, contains, errors, register, hex, features, sector, cylinder, measured, printed, ddd, hh, ss, sss, dd, millisec, quot, wraps, bf, fd, fc, cf, af, ef, reserved, native, ext, num, test_description, remaining, lba_of_first_error, span, min_lba, max_lba, current_test_status, not_testing, flags, scanning, selected, spans, remainder, pending, resume,