0

I have been working to migrate our current single instance database to a new clustered database running MySQL cluster.

It is a large database (several billion records) and, while it seems to be working reasonably well, I am having difficulty restoring a backup (for a second site development replica)

The backup contains only about 800 million reports, which the hardware should be able to handle fine. However, when I try to restore the backup (which can take hours to days!) some of the node restores just stop - for no apparent reason and with nothing obvious in the logs.

I have searched Google the best I can and can't seem to find anyone who has experienced this issue.

The database in question contains about 30 tables, one of which contains most of the reports. I can restore all of the tables' metadata fine, and all but the large table (using the exclude-table flag). But when I try to restore the large table I get this problem where ndb_restore just stops.

I am using MySQL cluster 5.6.23 with ndb-7.4.5 The cluster is built with 6 data nodes (running ndbmtd), 1 management node and 3 API nodes (each with 3 connections so logically 9 API nodes on 3 servers)

All the tables involved are disk data tables, the tablespace is large enough to contain the entire data-set, and the system has sufficient RAM to hold the indexes and indexed columns.

Any help with this would be greatly appreciated (if you need more information please ask!)

1 Answer 1

0

I think that I have found the solution.

I have been running the ndb_restore via a script which I have been executing through a SSH connection as I do not currently have physical access to the servers. As such, when the SSH session dies, ndb_restore received a SIGHUP. I believe that this is what has been causing the process to stop without any notice or log messages.

I have since found that I can prevent this by, once the script has been backgrounded (ctrl+z; bg), running:

disown [job id]

For details on this see:

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .