1

This question already has an answer here:

I'm running MongoDB 4.0 with the WiredTiger storage engine under CentOS 7. The data files are held on a mounted XFS volume.

I can't use my hosting provider's backup service because it doesn't support XFS volumes. This means I need to take snapshots some other way and copy them over to a redundant file system that does have backups enabled.

I've read this article which gives various options for taking the snapshots, but it doesn't go into detail about how to ensure a Valid Database at the Time of Snapshot.

In addition to the LVM method described I've also looked at xfsdump and have even considered rsync (because the data comprises many small files). But regardless of the snapshot/copy method, how do I ensure that the backup is in a consistent state?

marked as duplicate by John Mahowald, Thomas, Community May 10 at 13:18

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

0

Only one way to ensure a consistent backup, restore it and spot check the data. Naturally this requires procedures to take a consistent backup first...


Databases are always being written to. Filesystem snapshot methods require taking an atomic snapshot of the volume across all of its LUNs. Only storage system layer or LVM layer can do this. (Not copying files.)

Consistency behaviors are specific to the DBMS used. From that link:

For the WiredTiger storage engine, the data files reflect a consistent state as of the last checkpoint. Checkpoints occur with every 2 GB of data or every minute.

These checkpoints are persisted to disk with the usual I/O system calls (probably pwrite() and fsync()). If the storage system has a volatile cache and it goes down, writes may be lost.

Journal allows you to take a snapshot at any time, inconsistencies will be fixed during journal replay. However, if your journal volume is different from your data volume, or you don't journal, read the bit about write lock before taking a snapshot.

In summary you need:

  • atomic snapshots across all LUNs
  • journal
  • a storage system where acknowledged writes are actually committed
  • Thanks. I am journaling, and it's on the same volume. – Tim May 9 at 18:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.