In a RAID array I had a have periodically lost a drive here and there over the past several months. Iwas always able to readd and resync without losing data. However at some point it looks like some minor corruption happened and this makes DRBD unhappy.
Using fsck did not help either.
Dec 19 06:01:45 storageboxtest4 kernel: [19005.945890] EXT3-fs error (device drbd0): ext3_get_inode_loc: unable to read inode block - inode=22184379........
Before getting into the output here is my typical experience with SMART, there is what I call a "bad disk" with pending and uncorrectable sectors that cannot be reallocated.
It has caused a kernel panic and system crash repeatedly as we can see from the logs.
But SMART says it has "PASSED" its self assessment. SMART is still useful to me but it is more about looking at Current_Pending_Sector.
Any time I have had anything but 0 for that attribute it........
Here is the only solution Ifound that works in Firefox:
*Note you could basically just copy all of the commands in bold (the rest just shows the output)
Install sqlite3
sqlite3 places.sqlite
sqlite> .clone places.sqlite-fixed
moz_places... done
moz_historyvisits... done
moz_inputhistory..........
The easiest way is to use SSHand DD or a combination of netcat. SSHwill be a little slower due to encryption but is the most secure way (on two older systems the average clone speed is about 40-50MB/s). This is also OS independent as it doesn't matter what the source OS is because you are literallly cloning the drive so you retain the partition table and settings.
Clone HDD using SSH and DD........
This happened on Centos for no apparent reason with no obvious issue in the logs. Data could be read fine but not written (possibly due to some corruption or out of memory issue in the OpenVZ container is the best guess).
All mysql update and insert queries failed freezing without any error log on any database and table.
Tried to restart:
service mysqld restart
Timeout error occurred trying to stop MySQL........
The solution is simple but strange, if you copy your /var/lib/mysql directory to another server and think it will work, be sure to check if you have /var/log/mysql and binary log files. If you do, the server will not work and will give you errors like below and crash without the proper log files.
UPDATE user SET password=password("newpass") WHERE user='root';
flush privileges;
ERROR 2013 (HY000): Lost connection to MySQL server durin........
This happened while trying to delete several thousand users from phpBB and basically corrupted the innoDB tables.
InnoDB: Page lsn 3 881164362, low 4 bytes of lsn at page end 881164362
InnoDB: Page number (if stored to page already) 86920,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be an index page where index id is 700
InnoDB: (index "tid_post_time" of table "phpBBdb2005"."phpbb3_p........
This is a great way to test and experiment without risking your current table, or to make a backup of your current table to avoid corruption or dataloss due to malicious activity eg. deletion etc..
There are two steps and two commands
1.)Copy Table Structure
"cars_backup" is the new table and it will be a copy of the table called "cars"........
I backed up everything in the /mnt/sd_card directory thinking that some dataloss could occur for some reason but purposely left my microSDHC unbacked up thinking that "it won't touch that since it's external" and Samsung's and other manufacturers website even say this (that it won't be affected and not to worry etc).
Apparently I was wrong, my microSD was "undetected" and asked to be formatted after the upgrade (there goes 3-months worth of family photos). No........
Timeout error occurred trying to stop MySQL Daemon.
Stopping MySQL: [FAILED]
[root@devbox stats]#
Solution (force kill mysqld - warning you do risk table corruption here but there's no choice i........
mysql errors even though these files do exist:
110405 13:21:37 InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory.
InnoDB: File name ./ibdata1
InnoDB: File operation call: 'open'.
InnoDB: Cannot continue operation.
110405 13:26:15 InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means my........
This doesn't seem to be widely known (maybe it's in some documentation that none of us read though)but there's an easy way to check the integrity of any mdadm array:
sudo echo check > /sys/block/md0/md/sync_action
-bash: /sys/block/md0/md/sync_action: Permission denied
sudo will never work, this only works as root since echo is not actually a binary/command. It is built-into bash.
/sys/devices/virtu........
This really made me nervous but notice the mdstat says "check". This is because in Ubuntu there is a scheduled mdadm cronscript that runs everyday on Sunday at 00:57 that checks your entire array. This is a good way because it prevents gradual but unnoticed data corruption which Inever thought of.
As long as the check completes properly you have peace of mind knowing that your data integretiy is assured and that your hard drives are functioning properly (I'........
I've gotten this error enough to bother posting about it, because I've come across so many servers where this happens, so what could "Error 28" possibly mean? Is your database corrupt, or is this a sign of a RAID failure/corruption or even worse, bad blocks on a clients system who has no RAID and never took backups?
No, check your free blocks, it simply means you have no space. This was the result of a script that was overzealous and backed up the entire database........
Backing MySQL Databases
Backing Up/Dumping All Mysql Databases To A Single File
mysqldump --all-databases -u admin -p > allmysqldatabases.sql
The "-all-databases" clause is pretty obvious isn't it? It means that it will backup all databases.
The "-u admin" means login using the user "admin", if you h........