0

Today we got some DB crash. The DB is InnoDB. At firstin log:

120404 10:57:40  InnoDB: ERROR: the age of the last checkpoint is 9433732,
InnoDB: which exceeds the log group capacity 9433498.
InnoDB: If you are using big BLOB or TEXT rows, you must set the
InnoDB: combined size of log files at least 10 times bigger than the
InnoDB: largest such row.
120404 10:58:48  InnoDB: ERROR: the age of the last checkpoint is 9825579,
InnoDB: which exceeds the log group capacity 9433498.
InnoDB: If you are using big BLOB or TEXT rows, you must set the
InnoDB: combined size of log files at least 10 times bigger than the
InnoDB: largest such row.
120404 10:59:04  InnoDB: ERROR: the age of the last checkpoint is 13992586,
InnoDB: which exceeds the log group capacity 9433498.
InnoDB: If you are using big BLOB or TEXT rows, you must set the
InnoDB: combined size of log files at least 10 times bigger than the
InnoDB: largest such row.
120404 10:59:20  InnoDB: ERROR: the age of the last checkpoint is 18059881,
InnoDB: which exceeds the log group capacity 9433498.
InnoDB: If you are using big BLOB or TEXT rows, you must set the
InnoDB: combined size of log files at least 10 times bigger than the
InnoDB: largest such row.

after manual service stop and normal PC restart :

120404 11:12:35  InnoDB: Error: page 3473451 log sequence number 105 802365904
InnoDB: is in the future! Current system log sequence number 105 796344770.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: for more information.
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 1 row operations to undo
InnoDB: Trx id counter is 0 1103869440
120404 11:12:37  InnoDB: Error: page 0 log sequence number 105 834817616
InnoDB: is in the future! Current system log sequence number 105 796344770.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: for more information.
InnoDB: Last MySQL binlog file position 0 3710603, file name .\mysql-bin.000336
InnoDB: Starting in background the rollback of uncommitted transactions
120404 11:12:38  InnoDB: Rolling back trx with id 0 1103866646, 1 rows to undo
120404 11:12:38  InnoDB: Started; log sequence number 105 796344770
120404 11:12:38  InnoDB: Error: page 2097163 log sequence number 105 803249754
InnoDB: is in the future! Current system log sequence number 105 796344770.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: for more information.

InnoDB: Rolling back of trx id 0 1103866646 completed
120404 11:12:39  InnoDB: Rollback of non-prepared transactions completed
120404 11:12:39 [Note] Event Scheduler: Loaded 0 events
120404 11:12:39 [Note] wampmysqld: ready for connections.
Version: '5.1.53-community'  socket: ''  port: 3306  MySQL Community Server (GPL)
120404 11:12:40  InnoDB: Error: page 2097162 log sequence number 105 803215859
InnoDB: is in the future! Current system log sequence number 105 796345097.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: for more information.
120404 11:12:40  InnoDB: Error: page 2097156 log sequence number 105 803181181
InnoDB: is in the future! Current system log sequence number 105 796345097.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: for more information.
120404 11:12:40  InnoDB: Error: page 2097157 log sequence number 105 803193066
InnoDB: is in the future! Current system log sequence number 105 796345097.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: for more information.

when tried to recover data get : 

key_buffer_size=16777216
read_buffer_size=262144    
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 133725 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
0000000140262AFC    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402AAFA1    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402AB33A    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
0000000140268219    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000014027DB13    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402A909F    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402A91B6    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000014025B9B0    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000014022F9C6    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
0000000140219979    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000014009ABCF    mysqld.exe!?ha_initialize_handlerton@@YAHPEAUst_plugin_int@@@Z()
000000014003308C    mysqld.exe!?plugin_lock_by_name@@YAPEAUst_plugin_int@@PEAVTHD@@PEBUst_mysql_lex_string@@H@Z()
00000001400375A9    mysqld.exe!?plugin_init@@YAHPEAHPEAPEADH@Z()
000000014001DACE    mysqld.exe!handle_shutdown()
000000014001E285    mysqld.exe!?win_main@@YAHHPEAPEAD@Z()
000000014001E632    mysqld.exe!?mysql_service@@YAHPEAX@Z()
00000001402EA477    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402EA545    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000007712652D    kernel32.dll!BaseThreadInitThunk()
000000007725C521    ntdll.dll!RtlUserThreadStart()
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
120404 14:17:49 [Note] Plugin 'FEDERATED' is disabled.
120404 14:17:49 [Warning] option 'innodb-force-recovery': signed value 8 adjusted to 6
InnoDB: The user has set SRV_FORCE_NO_LOG_REDO on
InnoDB: Skipping log redo
InnoDB: Error: trying to access page number 4290979199 in space 0,
InnoDB: space name .\ibdata1,
InnoDB: which is outside the tablespace bounds.
InnoDB: Byte offset 0, len 16384, i/o type 10.
InnoDB: If you get this error at mysqld startup, please check that
InnoDB: your my.cnf matches the ibdata files that you have in the
InnoDB: MySQL server.
120404 14:17:52  InnoDB: Assertion failure in thread 3928 in file .\fil\fil0fil.c lin23
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
120404 14:17:52 - mysqld got exception 0xc0000005 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 133725 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
0000000140262AFC    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402AAFA1    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402AB33A    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
0000000140268219    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000014027DB13    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402A909F    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402A91B6    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000014025B9B0    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000014022F9C6    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
0000000140219979    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000014009ABCF    mysqld.exe!?ha_initialize_handlerton@@YAHPEAUst_plugin_int@@@Z()
000000014003308C    mysqld.exe!?plugin_lock_by_name@@YAPEAUst_plugin_int@@PEAVTHD@@PEBUst_mysql_lex_string@@H@Z()
00000001400375A9    mysqld.exe!?plugin_init@@YAHPEAHPEAPEADH@Z()
000000014001DACE    mysqld.exe!handle_shutdown()
000000014001E285    mysqld.exe!?win_main@@YAHHPEAPEAD@Z()
000000014001E632    mysqld.exe!?mysql_service@@YAHPEAX@Z()
00000001402EA477    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
00000001402EA545    mysqld.exe!?check_next_symbol@Gis_read_stream@@QEAA_ND@Z()
000000007712652D    kernel32.dll!BaseThreadInitThunk()
000000007725C521    ntdll.dll!RtlUserThreadStart()
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

Any suggestion how to get DB working ?

Improve this question
4
  • did you run mysql.. change some of the inno settings in my.cnf then restart it to cause this?
    – Mike
    Apr 4, 2012 at 14:40
  • Mysql was running. Then it hang - no response , service stopped manually, then pc reboot....
    – martin
    Apr 4, 2012 at 14:53
  • do you care about the data that might already be in your database or can you start fresh?
    – Mike
    Apr 4, 2012 at 16:30
  • Data in db are super important for me...
    – martin
    Apr 4, 2012 at 19:52

2 Answers 2

Reset to default
0

start by making a backup of the physical files I'd say, even now that its causing problems.

it is preferable to use the standard mysql recovery tools if possible like mysqlcheck, but if it won't even let you get that far, if it comes down to it you may have to move the physical files for all the databases out of the data dir and then move one database back in at a time, restarting mysql each time, to see which database it crashes on. This doesn't solve the fact that you have at least one bad database or table, but it should let you get everything else back up and running as fast as possible if it helps any.

when in doubt, try http://dev.mysql.com/doc/refman/5.1/en/crashing.html

Improve this answer
1
  • I tried to move file away. Problem is, that everything is stored in one ibdata file around 65gb big, containing only one DB.... Also tried to start with innodb_force_recovery didn't worked. The ibdata file seems to be corrupted..
    – martin
    Apr 4, 2012 at 22:06
0

Percona saved our lives and data too. So if facing some problems with corrupted data try percona on linux to get your data...

Percona innodb data recovery tool

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .