On an Amazon EC2 server with Linux AMI, I currently have a daily backup cron job be run:
TMP_BACKUP_FILE="/tmp/backup.tar" BACKUP_FILE="/home/ec2-user/Dropbox/Backup/backup.tar" rm -f "$TMP_BACKUP_FILE" tar cf "$TMP_BACKUP_FILE" \ /home /root /var/lib/redis /var/spool /etc mv "$TMP_BACKUP_FILE" "$BACKUP_FILE" chown ec2-user:ec2-user "$BACKUP_FILE"
The tar dump is uploaded into Dropbox:
The file name is always
backup.tar. No time stamp is added. Dropbox takes care of versioning.
The tar file is uncompressed, with the idea that this facilitates Dropbox delta sync.
However, when monitoring (with
dropbox.py) upload times of
Dropbox, I get the impression that the Dropbox client is not using delta sync.
This is bad:
Without delta syncing, server bandwidth is wasted.
I have shared the backup folder with my personal Dropbox, and so every day
backup.taris downloaded to my laptop (and from there it goes into an offline backup system). Without delta syncing, downloads take long and again waste bandwidth.
What is a good backup archive format for the given purpose?
I could rsync to a loop mounted image file. Does that sound like a good idea?
I just did a test with the
rdiff utility, which is part of
librsync, and according to Wikipedia Dropbox depends on librsync. The test
shows that the delta is just 2.6 MB in size, thereby considerably smaller than
the 354 MB of the backup archive. So, perhaps tar is an OK format for the given
purpose. The test:
$ mv ~/Dropbox/Backup/backup.tar /tmp $ sudo ~/bin/backup.sh $ mv ~/Dropbox/Backup/backup.tar /tmp/backup_new.tar $ cd /tmp $ rdiff signature backup.tar >backup.tar.signature $ rdiff delta backup.tar.signature backup_new.tar >backup_new.tar.delta $ ls -lh backup.tar backup_new.tar backup_new.tar.delta -rw-rw-r-- 1 ec2-user ec2-user 354M Dec 21 13:39 backup_new.tar -rw-rw-r-- 1 ec2-user ec2-user 2.6M Dec 21 13:55 backup_new.tar.delta -rw-rw-r-- 1 ec2-user ec2-user 354M Dec 21 00:10 backup.tar
I asked in the Dropbox forum concerning how to find out the size of the delta that Dropbox uploads.