I started recovering data for a non-profit organization from a server that had a hard drive failure.

The mysqldump was not working, but I managed to tar the /var/lib/mysql on time and download via ssh.

I also dumped the list of packages (dpkg -l) from the server, so I am able to run the same MySQL version.

When I tried to start dockerized MySQL 5.5.45 with /var/lib/mysql mounted from previous server (with valid permissions) I got a stack trace from InnoDB.

I successfully recovered MyISAM tables, and I noticed that the ibdata1 is the problem there.

db_mysql_1  | InnoDB: End of page dump
db_mysql_1  | 180822 17:14:10  InnoDB: Page checksum 1575996416, prior-to-4.0.14-form checksum 1371122432
db_mysql_1  | InnoDB: stored checksum 0, prior-to-4.0.14-form stored checksum 0
db_mysql_1  | InnoDB: Page lsn 0 0, low 4 bytes of lsn at page end 0
db_mysql_1  | InnoDB: Page number (if stored to page already) 0,
db_mysql_1  | InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 0
db_mysql_1  | InnoDB: Page may be a freshly allocated page
db_mysql_1  | 17:14:10 UTC - mysqld got signal 11 ;
db_mysql_1  | This could be because you hit a bug. It is also possible that this binary
db_mysql_1  | or one of the libraries it was linked against is corrupt, improperly built,
db_mysql_1  | or misconfigured. This error can also be caused by malfunctioning hardware.
db_mysql_1  | We will try our best to scrape up some info that will hopefully help
db_mysql_1  | diagnose the problem, but since we have already crashed, 
db_mysql_1  | something is definitely wrong and this may fail.
db_mysql_1  | 
db_mysql_1  | key_buffer_size=8388608
db_mysql_1  | read_buffer_size=131072
db_mysql_1  | max_used_connections=0
db_mysql_1  | max_threads=151
db_mysql_1  | thread_count=0
db_mysql_1  | connection_count=0
db_mysql_1  | It is possible that mysqld could use up to 
db_mysql_1  | key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 338512 K  bytes of memory
db_mysql_1  | Hope that's ok; if not, decrease some variables in the equation.
db_mysql_1  | 
db_mysql_1  | Thread pointer: 0x0
db_mysql_1  | Attempting backtrace. You can use the following information to find out
db_mysql_1  | where mysqld died. If you see no messages after this, something went
db_mysql_1  | terribly wrong...
db_mysql_1  | stack_bottom = 0 thread_stack 0x40000
db_mysql_1  | mysqld(my_print_stacktrace+0x35)[0x7aa245]
db_mysql_1  | mysqld(handle_fatal_signal+0x403)[0x677123]
db_mysql_1  | /lib/x86_64-linux-gnu/libpthread.so.0(+0xf890)[0x7fafa2cdc890]
db_mysql_1  | mysqld[0x8a7b5a]
db_mysql_1  | mysqld[0x824f51]
db_mysql_1  | mysqld[0x828c46]
db_mysql_1  | mysqld[0x855f49]
db_mysql_1  | mysqld[0x85ad20]
db_mysql_1  | mysqld[0x845321]
db_mysql_1  | mysqld[0x7f1778]
db_mysql_1  | mysqld[0x7c0f0d]
db_mysql_1  | mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x48)[0x679f98]
db_mysql_1  | mysqld[0x586dca]
db_mysql_1  | mysqld(_Z11plugin_initPiPPci+0xafa)[0x58a8fa]
db_mysql_1  | mysqld[0x50912b]
db_mysql_1  | mysqld(_Z11mysqld_mainiPPc+0x3ea)[0x509cfa]
db_mysql_1  | /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fafa1dfdb45]
db_mysql_1  | mysqld[0x4fee3a]
db_mysql_1  | The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
db_mysql_1  | information that should help you find out what is causing the crash.

And resolved stack:

root@fbda086ffd7e:/# resolve_stack_dump -s ./mysql.sym -n ./mysql.stack 
0x7aa245 my_print_stacktrace + 53
0x677123 handle_fatal_signal + 1027
0x7f74a5510890 _end + -1537442712
0x8a7b5a dict_table_check_foreign_keys + 343050
0x824f51 _ZN21ha_innobase_add_indexD0Ev + 391329
0x828c46 _ZN21ha_innobase_add_indexD0Ev + 406934
0x855f49 dict_table_check_foreign_keys + 8185
0x85ad20 dict_table_check_foreign_keys + 28112
0x845321 _ZN21ha_innobase_add_indexD0Ev + 523377
0x7f1778 _ZN21ha_innobase_add_indexD0Ev + 180424
0x7c0f0d innobase_convert_to_system_charset + 51533
0x679f98 _Z24ha_initialize_handlertonP13st_plugin_int + 72
0x586dca _Z16check_valid_pathPKcm + 4938
0x58a8fa _Z11plugin_initPiPPci + 2810
0x50912b kill_server_thread + 587
0x509cfa _Z11mysqld_mainiPPc + 1002
0x7f74a4631b45 _end + -1553035491
0x4fee3a _start + 42

Is there anything I can do to recover data from that ibdata1?

Thanks and I wish you a good day.

  • could try a later 5.5 version on a copy of the data to see if recovers it better. – danblack Aug 23 '18 at 6:22
  • I already tried, no luck. I also tried to run Percona database, no luck, just a longer stack trace. – Krzysztofa Krzysztof Aug 23 '18 at 11:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.