Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

On an Amazon EC2 server with Linux AMI, I currently have a daily backup cron job be run:

TMP_BACKUP_FILE="/tmp/backup.tar"
BACKUP_FILE="/home/ec2-user/Dropbox/Backup/backup.tar"

rm -f "$TMP_BACKUP_FILE"
tar cf "$TMP_BACKUP_FILE" \
    /home /root /var/lib/redis /var/spool /etc
mv "$TMP_BACKUP_FILE" "$BACKUP_FILE"
chown ec2-user:ec2-user "$BACKUP_FILE"

The tar dump is uploaded into Dropbox:

  • The file name is always backup.tar. No time stamp is added. Dropbox takes care of versioning.

  • The tar file is uncompressed, with the idea that this facilitates Dropbox delta sync.

However, when monitoring (with dropbox.py) upload times of backup.tar into Dropbox, I get the impression that the Dropbox client is not using delta sync. This is bad:

  1. Without delta syncing, server bandwidth is wasted.

  2. I have shared the backup folder with my personal Dropbox, and so every day backup.tar is downloaded to my laptop (and from there it goes into an offline backup system). Without delta syncing, downloads take long and again waste bandwidth.

What is a good backup archive format for the given purpose?

I could rsync to a loop mounted image file. Does that sound like a good idea?

Update

I just did a test with the rdiff utility, which is part of librsync, and according to Wikipedia Dropbox depends on librsync. The test shows that the delta is just 2.6 MB in size, thereby considerably smaller than the 354 MB of the backup archive. So, perhaps tar is an OK format for the given purpose. The test:

$ mv ~/Dropbox/Backup/backup.tar /tmp
$ sudo ~/bin/backup.sh
$ mv ~/Dropbox/Backup/backup.tar /tmp/backup_new.tar
$ cd /tmp
$ rdiff signature backup.tar >backup.tar.signature
$ rdiff delta backup.tar.signature backup_new.tar >backup_new.tar.delta
$ ls -lh backup.tar backup_new.tar backup_new.tar.delta
-rw-rw-r-- 1 ec2-user ec2-user 354M Dec 21 13:39 backup_new.tar
-rw-rw-r-- 1 ec2-user ec2-user 2.6M Dec 21 13:55 backup_new.tar.delta
-rw-rw-r-- 1 ec2-user ec2-user 354M Dec 21 00:10 backup.tar

I asked in the Dropbox forum concerning how to find out the size of the delta that Dropbox uploads.

share|improve this question

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.