Say I am running an HTTP server with data at /var/www. I want to backup /var/www to /root/backup/.tmp/var/www (and then tar them to somewhere) daily automatically.

Mostly the backup is using rsync technique. The problem is that since the HTTP server is running, there could be file modification during an rsync backup process.

For an HTTP server a certain "transaction" could involve multiple files, e.g. modifying file A and B at once, and therefore such scenario is possible: rsync backups file A => a transaction occurs and file A and B are modified => rsync backups file B. This causes the backup-ed files to be inconsistent (A is before transaction while B is after transaction).

For an HTTP server shutting down for backup is mostly not viable. Is there a way to avoid such inconsistent file backup?

This also applies to potentially other services like FTP (a certain "transaction" could be an upload of a folder which contains multiple files) or so.

  • That's why many people flush write caches to disk before making a snapshot (with a volume manager or from their storage array) and run consistent file level backups using that snapshot. – HBruijn Aug 9 '16 at 11:04

It depends on your environment, but basically it is not possible with rsync only. If your filesystem supports snapshots, use it for freeze whole filesystem state in one point in time and backup from that snapshot. If it is not possible, you have to look if inconsistency is really problem (it isn't in 90% of environments I've seen) and if yes, you will need to shut down new connections for this server during backups. It should be no problem, because your customers will be served from another server in your pool (if you have only one server, you have bigger issue that inconsistent backups).

| improve this answer | |
  • A filesystem snapshot takes zero time so that it's impossible that a file be modified during the snapshot process? – Danny Lin Aug 9 '16 at 16:00
  • My case is one server only, so shutting down the service for backup is not viable. Currently my strategy is to backup regularly and let my PC download the backup archives automatically when it's online. – Danny Lin Aug 9 '16 at 16:03
  • Yep, filesystem snapshot is atomic. It works just as freezing "inodes" and new modifications are saved into separated. You have to make backups with filesystem tools but it is safe. See btrfs or zfs for more info. – Ondra Sniper Flidr Aug 9 '16 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.