0

I have been working to migrate our current single instance database to a new clustered database running MySQL cluster.

It is a large database (several billion records) and, while it seems to be working reasonably well, I am having difficulty restoring a backup (for a second site development replica)

The backup contains only about 800 million reports, which the hardware should be able to handle fine. However, when I try to restore the backup (which can take hours to days!) some of the node restores just stop - for no apparent reason and with nothing obvious in the logs.

I have searched Google the best I can and can't seem to find anyone who has experienced this issue.

The database in question contains about 30 tables, one of which contains most of the reports. I can restore all of the tables' metadata fine, and all but the large table (using the exclude-table flag). But when I try to restore the large table I get this problem where ndb_restore just stops.

I am using MySQL cluster 5.6.23 with ndb-7.4.5 The cluster is built with 6 data nodes (running ndbmtd), 1 management node and 3 API nodes (each with 3 connections so logically 9 API nodes on 3 servers)

All the tables involved are disk data tables, the tablespace is large enough to contain the entire data-set, and the system has sufficient RAM to hold the indexes and indexed columns.

Any help with this would be greatly appreciated (if you need more information please ask!)

0

I think that I have found the solution.

I have been running the ndb_restore via a script which I have been executing through a SSH connection as I do not currently have physical access to the servers. As such, when the SSH session dies, ndb_restore received a SIGHUP. I believe that this is what has been causing the process to stop without any notice or log messages.

I have since found that I can prevent this by, once the script has been backgrounded (ctrl+z; bg), running:

disown [job id]

For details on this see:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.