MySQL Server crashes with message like following:
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page
To protect data InnoDB uses checksums (stored together with pages). Whenever InnoDB reads from disk, it calculates checksum of each page and compares it with checksum loaded alongside from disk. If values are different, something really wrong happened and InnoDB will shutdown MySQL Server to prevent further logical or physical corruption.
How to find out why corruption happened
There is no universal solution. Most typically there is some problem with hardware, e.g. physical disk or memory failure, bad driver/controller or even OS kernel bug. Below are some ideas:
- On Linux platforms sometimes resetting Page Cache solves the problem:
echo 2 > /proc/sys/vm/drop_caches
- Check system logs how any possible hardware malfunction.
- If InnoDB crashes every time with particular page, it most typically physical disk failure: run detailed disk diagnostics, specific to your OS/hardware.
- If crashes are random and not repeatable with same queries, it is possible that RAM is failing: run detailed RAM diagnostics.
- It may be beneficial to check InnoDB files with innochecksum utility when MySQL server is down.
How to recover from corruption
The most important is to perform detailed hardware diagnostics to eliminate chance of problem propagation. If OS I/O cache is the reason of corrupted disk reads, resetting cache or rebooting OS should help to eliminate immediate problem, and database may become operational again.
Sometimes the only solution is to back up data in force recovery mode.
Leave a Reply