rsync for backups: without --remove option and rotations…?

Question

Is reasonably safe to use rsync (without --delete option) to make my daily backups and keep only 1 copy on the backup server (instead of having multiple copies with rotations) ?

If I don't enable the --delete option, any backup file won't be ever deleted, correct ?

thanks

Answer 1

(this started as a comment on OnoeOfOne's answer, but grew too long for that...)

I use an rsync+cp -al method similar to that show in http://www.mikerubel.org/computers/rsync_snapshots/ (that page was my original crib sheet some time in 2005, and I've not modified things an awful lot since) both for my personal backups and for managing the online+onsite and online+offite backups at work.

How much space this takes depends on the balance of sizes and how often files are changed and deleted. For us, keeping daily snapshots for over a year this way doesn't take much more space than three full copies of the data, as most of our files on the general network shares are not modified or removed often. In fact for many of the snapshots the directory structure takes more space than the data in changed files.

One thing to be wary of is that because each copy of the same version of a file is in fact the same data, on-disk corruption could kill the file in all your snapshots in one fell swoop, so there is still a case for keeping multiple copies. We mitigate this by having multiple copies on different machines and all the machines using RAID1 to protect against certain possible physical problems. Another way to address this if you only have one online backup location is to actually keep two copies and sync them individually, or just force a full refresh occasionally (say, once a month or once a week) so you end up with groups of snapshots: i.e. for a monthly forced refresh-all all the identical files in snapshots for January are the same data block, as are they in February, but there are at least two copies of the actual data if it existed in both months.

Answer 2

If you don't enable --delete, files are not deleted yes. Although a file could be truncated to 0 bytes if that was what was at the source.

Also check out the --backup option to see if that is suitable for what are you doing.

How are you going to prevent data corruption from simply being copied and destroying your only backup copy, or are you archiving them off somewhere?

Depending on what the source looks like and how it changes, you may over time accumulate lots of cruft and old data this way if you never delete it.

Answer 3

Take a look at rsnapshot. It does an rsync for modified files, but creates a hardlink for files not modified since last rotation. It's almost as simple to use as raw rsync, but you can save this space you're worried about.

Answer 4

What I personally use is rsync using hard-links and keep 5 copies, the trick is, if the file doesn't change, it won't take extra space that way, and it makes it very easy to recover if something blows up.

#!/bin/sh
BACKUP_DIR=/mnt/data-3/backups/

cd ${BACKUP_DIR}
#remove the oldest backup
rm -rf backup.4 backup.4.log.bz2 &>/dev/null
recycle() {
        i=$1; y=$(($i+1))
        b=${2-backup}
        mv "${b}.$i" "${b}.$y" &>/dev/null
        mv "${b}.$i.log.bz2" "${b}.$y.log.bz2" &>/dev/null
}
recycle 3
recycle 2
recycle 1
recycle 0

OPTS="--numeric-ids --delete --delete-after --delete-excluded"
DIRS_TO_BACKUP="/home /var"
nice -n20 ionice -c2 -n2 rsync -axlHh -v --link-dest=../backup.1 ${OPTS} ${DIRS_TO_BACKUP} backup.0/ --exclude-from=/root/.rsync-exclude 2>&1 | bzip2 -9 > backup.0.log.bz2

my /root/.rsync-exclude :

*~
*.cmd*
*.log
cache4
/tmp/
.ccache
.thumbnails/
lost+found
/var/log/
/var/run/
/var/lock/
/var/tmp/
/usr/src/