3

I need to restore huge dump taken with mongodump utility. As the collections are big and mongorestore rebuilds indexes in backround, index building takes very long to finish. What I want is to somehow make mongorestore to build indexes in foreground so that the index building would take less time to finish.

I know I can use --noIndexRestore to ignore indexes but then I will have to build indexes one by one, which is a problem again. I am looking for the best way to speed up index building with mongorestore utility.

1 Answer 1

3

Only indexes that were created with {background: true} originally (and hence have that option set in the system.indexes collection) are built in the background once restored. Once you have the database dumped you will have your data in the name.bson file and the metadata (indexes to build) in the name.metadata.json file. Unlike the BSON file, the JSON metadata is just text and can be modified before you restore.

As a test, I created a collection foo.bar, with 2 additional indexes besides the default on the a field and on the b field. Here is what the dumped bar.metadata.json file looked like in the foo folder once I had dumped it out:

{ "indexes" : [ { "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "foo.bar" }, { "v" : 1, "key" : { "a" : 1 }, "name" : "a_1", "ns" : "foo.bar" }, { "v" : 1, "key" : { "b" : 1 }, "name" : "b_1", "ns" : "foo.bar", "background" : true } ] }

You can "fix" this with your favorite editor and remove , "background" : true with a find & replace or similar. Or you could use a sed one liner, something like:

 sed 's/, "background" : true//g' bar.metadata.json 

Which, as expected, gives us:

{ "indexes" : [ { "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "foo.bar" }, { "v" : 1, "key" : { "a" : 1 }, "name" : "a_1", "ns" : "foo.bar" }, { "v" : 1, "key" : { "b" : 1 }, "name" : "b_1", "ns" : "foo.bar" } ] }

I tested a restore with the above bar.metadata.json file and my indexes were no longer created in the background, and were not marked as such in system.indexes any longer.

3
  • I knew this method. Looks like this is the only way to avoid building indexes in background. Thanks a lot for the confirmation and for the effort to test it.
    – Abhay PS
    Dec 4, 2014 at 7:32
  • 1
    Unfortunately this does not work with dumps created using the --archive option... Nov 25, 2016 at 14:58
  • 2
    that option didn't exist when the question/answer was written - I would suggest a new question (probably on the DBA stack exchange) to figure out what it possible with archiving
    – Adam C
    Nov 28, 2016 at 20:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .