I have been working to migrate our current single instance database to a new clustered database running MySQL cluster.
It is a large database (several billion records) and, while it seems to be working reasonably well, I am having difficulty restoring a backup (for a second site development replica)
The backup contains only about 800 million reports, which the hardware should be able to handle fine. However, when I try to restore the backup (which can take hours to days!) some of the node restores just stop - for no apparent reason and with nothing obvious in the logs.
I have searched Google the best I can and can't seem to find anyone who has experienced this issue.
The database in question contains about 30 tables, one of which contains most of the reports. I can restore all of the tables' metadata fine, and all but the large table (using the exclude-table flag). But when I try to restore the large table I get this problem where ndb_restore just stops.
I am using MySQL cluster 5.6.23 with ndb-7.4.5 The cluster is built with 6 data nodes (running ndbmtd), 1 management node and 3 API nodes (each with 3 connections so logically 9 API nodes on 3 servers)
All the tables involved are disk data tables, the tablespace is large enough to contain the entire data-set, and the system has sufficient RAM to hold the indexes and indexed columns.
Any help with this would be greatly appreciated (if you need more information please ask!)