1

I recently had a performance issue with one of my collections.
On a whim (and thanks to this article), I decided to try compacting (as written in the official documentation).
This worked brilliantly. However, I'm now wondering how often I should do it.
Since compacting is not a completely online task (I can only do it on passive nodes), I can't decide to do it every night and forget about it.

I couldn't find any documentation about knowing when a DB/collection has a high fragmentation rate. Do you have any experience with determining the fragmentation level of a DB (other than benchmarking)?

Note: I'm talking about "internal" fragmentation as in spaces inside the file, not "external" fragmentation as in file spread accross the disk.

1

Compaction-level can be determined by comparing the datasize in a collection through db.stats. dataSize gives you how much data is in the collection, where storageSize tells you how big the files are. dataSize <= storageSize, but how big the difference is should tell you how much gain you'll get through compaction.

Mongo doesn't allow objects to not be entirely co-located, so you won't get cases where an object is scattered across the datafiles. Where this comes into play is if an object expands past it's free-allocation, the entire object has to be rewritten somewhere bigger.

When I was playing with Mongo databases, a compaction in a quarterly maintenance window was all we needed. But then, our dataset didn't have a whole lot of deletions, so we weren't creating voids that often. To figure out your rate, track those two dbStats values and see how they move over time.

  • I read that as well, but MongoDB preallocates a lot of space ahead anyway (something with 2^x), so how can I tell if the space is at the end or in tiny holes in the middle? – Nitz Jul 20 '14 at 13:16
  • @Nitz By seeing what your data looks like a day after compaction. That will give you your low-water mark. – sysadmin1138 Jul 20 '14 at 13:25
  • 1
    that's not technically true as that will only show preallocated space, not space that's "extra" allocated for each record (if that's what OP wanted to know). However padding factor in the same stats output will show if more than "default" is being allocated for each newly inserted record. – Asya Kamsky Jul 28 '14 at 4:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.