I have 2 servers running linux, one of them hosts picture sharing service (currently there's 193G of pictures) and I want to have incremental backup. Copying whole site every time is not an option so I want something like rsync but it should be able to use previous versions of backup and just hardlink up to date files. Is there some existing solution for this?
rsnapshot perfectly fits the bill here.
I'd recommend you
rdiff-backup, which is using the
rsync library. It creates incremental backups. The best thing is that the most recent backup is always stored as-is, so you can just use
scp etc. to restore data. If you need older data, there's an argument that lets you restore data.
We're using it to backup about 450GiB of data every night.
Example: Show backups:
root@xenbackup:~# rdiff-backup -l /backup/new/web1_example_org/ Found 26 increments: increments.2012-02-24T14:38:25+01:00.dir Fri Feb 24 14:38:25 2012 ... increments.2012-03-14T07:00:55+01:00.dir Wed Mar 14 07:00:55 2012 increments.2012-03-15T06:53:51+01:00.dir Thu Mar 15 06:53:51 2012 increments.2012-03-16T06:54:03+01:00.dir Fri Mar 16 06:54:03 2012 increments.2012-03-17T07:35:28+01:00.dir Sat Mar 17 07:35:28 2012 increments.2012-03-18T06:23:44+01:00.dir Sun Mar 18 06:23:44 2012 increments.2012-03-19T06:16:30+01:00.dir Mon Mar 19 06:16:30 2012 increments.2012-03-20T06:17:56+01:00.dir Tue Mar 20 06:17:56 2012 Current mirror: Wed Mar 21 06:20:32 2012
Example: Back up folder /var/www on web1.example.org to local /var/backups:
root@xenbackup:~# rdiff-backup email@example.com::/var/www /var/backups
There's some cool documentation here.
rdiff-backup creates a subdirectory called
rdiff-backup-data where it stores its incremental data and logs. You can even tell it to delete data older than 2weeks or so. Check the docs, this thing is amazing! :-)
Just for completeness sake, dirvish is a great tool that performs this task as well.