1

I've a web server that's offline at this moment because MySQL fails to start with an exception. I can't find the root cause of the problem, but I'm afraid of database corruption crashing the MySQL service.

Here are the /var/log/mysql/mysqld.err output:

==> /var/log/mysql/mysqld.err <==
2015-02-11 23:45:11 25171 [Warning] No argument was provided to --log-bin, and --log-bin-index was not used; so replication may break when this MySQL server acts as a master and has his hostname changed!! Please use '--log-bin=mysqld-bin' to avoid this problem.
2015-02-11 23:45:11 25171 [Note] InnoDB: Using atomics to ref count buffer pool pages
2015-02-11 23:45:11 25171 [Note] InnoDB: The InnoDB memory heap is disabled
2015-02-11 23:45:11 25171 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-02-11 23:45:11 25171 [Note] InnoDB: Memory barrier is not used
2015-02-11 23:45:11 25171 [Note] InnoDB: Compressed tables use zlib 1.2.8
2015-02-11 23:45:11 25171 [Note] InnoDB: Using Linux native AIO
2015-02-11 23:45:11 25171 [Note] InnoDB: Using CPU crc32 instructions
2015-02-11 23:45:11 25171 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2015-02-11 23:45:11 25171 [Note] InnoDB: Completed initialization of buffer pool
2015-02-11 23:45:11 25171 [Note] InnoDB: Highest supported file format is Barracuda.
2015-02-11 23:45:11 25171 [Note] InnoDB: The log sequence numbers 43953950155 and 43953950155 in ibdata files do not match the log sequence number 43955508491 in the ib_logfiles!
2015-02-11 23:45:11 25171 [Note] InnoDB: Database was not shutdown normally!
2015-02-11 23:45:11 25171 [Note] InnoDB: Starting crash recovery.
2015-02-11 23:45:11 25171 [Note] InnoDB: Reading tablespace information from the .ibd files...
2015-02-11 23:45:11 25171 [Note] InnoDB: Restoring possible half-written data pages 
2015-02-11 23:45:11 25171 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Last MySQL binlog file position 0 107020, file name ./mysqld-bin.000052
01:45:11 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=0
max_threads=151
thread_count=0
connection_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 134417 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x3c)[0x7f122fe0d18c]
/usr/sbin/mysqld(handle_fatal_signal+0x3c9)[0x7f122fc00c29]
/lib64/libpthread.so.0(+0x102b0)[0x7f122e4ef2b0]
/usr/sbin/mysqld(+0x792dd9)[0x7f122ff7bdd9]
/usr/sbin/mysqld(+0x799787)[0x7f122ff82787]
/usr/sbin/mysqld(+0x71a1bd)[0x7f122ff031bd]
/usr/sbin/mysqld(+0x66f175)[0x7f122fe58175]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x41)[0x7f122fb65f41]
/usr/sbin/mysqld(+0x487c32)[0x7f122fc70c32]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0x5fc)[0x7f122fc7425c]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x7cb)[0x7f122fb5e7eb]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f122db379f5]
/usr/sbin/mysqld(+0x36c3d8)[0x7f122fb553d8]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

This crash happen on mysql-5.6.22.

Any idea of what's happening?

1
  • 2
    Your fastest way to resolution is to stand up a new database server and restore from a known-good backup. After doing that, you can spend time figuring out what happened here.
    – EEAA
    Feb 12, 2015 at 2:18

1 Answer 1

2

You have a (really badly) corrupt database, which is crashing mysql.

As a sidenote, this should only happed in rare cases like kernel freeze or power loss coupled with forcing innodb_flush_log_at_trx_commit to 2 or using a raid controller with write cache forced on but no backup battery.

If you have to recover the data, you can try starting mysql with innodb_force_recovery set to a non-zero value - first try to set it to 1, see if mysql starts, if so do a mysqldump of the database; if a value of 1 does not help, try with 2 and so on.

Further details are in the mysql documentation:

http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html

1
  • I will take a look on the link. And yes. We had an extended power outage and the machine went down. The RAID is done by mdadm on Linux, so it's a software solution. Things are probably corrupted. Feb 12, 2015 at 21:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .