This happened while an mdadm array was syncing, all access from writing a new blank file to opening a small .txt file was very slow:
[222117.312078] kjournald starting. Commit interval 5 seconds
[222117.685060] EXT3-fs (md0): using internal journal
[222117.685096] EXT3-fs (md0): mounted filesystem with ordered data mode
[222122.376847] kjournald starting. Commit interval 5 seconds
[222122.602825] EXT3-fs (md2): using internal jour........
These were caused by a bad stick of Corsair RAM
[] free_hot_cold_page+0xfc/0x150
[] __pagevec_free+0x14/0x1a
[] release_pages+0x127/0x12f
[] __pagevec_release+0x15/0x1d
[] __invalid_mapping_pages+0x120/0x156
[........
I had a system running a 128MB live CD image with 2.8 gigs of available RAM and the OOM kernel killer went crazy when using dd for more than 8 minutes and kept killing everything. I've read that this is due to a low-memory issue and paging in the kernel and 32-bit systems with lots of RAM.
I even enabled swapspace on my LiveCD and the issue happened 25 minutes into dd rather than 8 minutes, so what gives?
Also no swap space was ever used!
cat /proc/s........
I like dd, although it only reads it, usually a read test of the entire disk will uncover if your hard drive is bad in some parts. This is a good thing to do at least once a month, a lot of times bizarre program behavior, laginess and crashing/unnmounting problems etc.. are due to a failing disc and SMART won't know it or indicate a problem:
We must also remember there's never a guarantee, I've found that ever since we moved to larger and more platters per drive with 1TB drives........
Jan 16 04:02:03 centosbox syslogd 1.4.1: restart.
Jan 16 04:07:34 centosbox kernel: INFO: task updatedb:20771 blocked for more than 300 seconds.
Jan 16 04:07:34 centosbox kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 16 04:07:34 centosbox kernel: updatedb D F78BE050 6476 20771 20766&n........
Out of memory: kill process 7559 (rsync) score 635 or a child
Killed process 7559 (rsync)
I was surprised to see this in my dmesg whenmy rsync backup suddenly stalled/stopped.
This system has 3 gigs of RAM and lots of free memory so I don't understand what is happening.
rsync invoked oom-killer: gfp_mask=0x200d2, order=0, oomkilladj=0
Pid: 7600, comm: rsync Not tainted 2.6.24.2 #83
[] oom_kill_pr........