0

I'm trying to rsync a single directory of files and store timestamped revisions at the destination. This is what I'm using at the moment:

rsync --verbose --progress --stats --compress --no-whole-file --inplace --recursive \
  --times --perms --owner --group --links --delete --backup --suffix=.`date +%y%m%d_%H%M` \
  user@remote:/backups/mysql/ /data/backups/mysql/remote/

The files at user@remote:/backups/mysql/ are like:

db1.gz
db2.gz

When I first ran rsync, I got the two files replicated at /data/backups/mysql/remote/ as expected. When I ran a second time after the files had changed, the old files got timestamped and the new files transferred to dest:

db1.gz.150522_1328
db2.gz.150522_1328
db1.gz
db2.gz

That all seems like I wanted it to, except when I ran it a third time (with changed files) the output showed 'deleting' messages before the sync:

deleting db1.gz.150522_1328
deleting db2.gz.150522_1328

And I was left with this at dest:

db1.gz.150522_1328.150522_1515
db2.gz.150522_1328.150522_1515
db1.gz.150522_1515
db2.gz.150522_1515
db1.gz
db2.gz

How come it has added the suffix to the existing backup files? Another thing that I need to get to the bottom of is that when I rsync there is no speedup, even given that the files are 99% the same and I'm using --rsyncable when I gzip them. (Is it renaming the backup files before delta-transfer?)

  • Actually, I guess what's happening is that it's looking at both the locations and because the backup files on the destination side aren't on the origin side, it's thinking they've been deleted and therefore deleting and backing them up again. The docs say: if --delete is also in effect (without --delete-excluded), rsync will add a "protect" filter-rule for the backup suffix to the end of all your existing excludes but this doesn't appear to be working. I will try a manual protect rule on Monday .. – Mike Campbell May 22 '15 at 16:19
  • you may want to have a look at rsnapshot. – lxg May 22 '15 at 22:49
1

As you already wrote it at the comment to your question, the previous backups are backuped again because the suffix changed and rsync assumes these are files that don't exist anymore on the source. You can avoid that by adding a static part to your suffix and use it as a filter, for example: --suffix=$(date +.backup~%y%m%d_%H%M) --filter="P *.backup~*"

Alternatively you can use --backup-dir=DIR to put the backups in a separate directory. However if the name of that directory is dynamic as well (e.g. --backup-dir=$(date +%F)), then --inplace will make your command less efficient.

I think the dynamic backup suffix is also the reason you don't notice a speedup: rsync does not find the corresponding backups to your files. Removing the "--inplace" parameter should help however (if you don't need it's advantage, that "Hard links are not broken").

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.