Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am researching a remote off-site backup strategy. The major limitation is that the upstream pipe is only ~50 kb/s, so with that in mind the incremental backup capabilities using file differentials of rdiff-backup seemed like the proper tool.

In a test case:

  1. First rdiffbackup was run on a local directory to a different local directory. In the directory was a 10GB pax archive of a Maildir directory.

  2. The 10GB pax archive was replaced with a 12GB pax archive in the source directory (representing a month's worth of additional emails). rdiff-backup was run again. I expected rdiff-backup to be faster this time because there was only a 2GB difference in file size, however, not only was the runtime longer but in the target directory there were 2 files, the original 10GB pax archive and a 12GB temp file.

Can rdiff-backup incrementally backup compressed archive files? It doesn't seem to be able to.

Currently the mail server writes pax Maildir backups to an external drive. But instead of using these as the rdiff-backup source, would it be better to let rdiff-backup just backup /home/%user%/Maildir (many, many small files)?

I suppose that if the external drive were to fail it would be better not to cripple the 2nd backup system as well!

cheers,

kilolima

edit : couldn't add a tag for 'pax'?!

share|improve this question

1 Answer 1

You're better off to not compress the data.

rdiff-backup does analyze files for differences, but if they're compressed archives, it may not be able to find any differences and thus be forced to store the entire new file again.

Also, you can use ssh -C to compress the ssh connection and save some bandwidth.

Finally, if possible, you should get some more bandwidth; that's barely better than dialup (or maybe it is dialup?). Backing up 12GB of data would take weeks over dialup, and even the 2GB difference could take days.

share|improve this answer
    
It's 50 KILObytes per second. Dial-up is about 5 kb/s. There is no way to increase upload speeds without paying hundreds of more $$ a month. And only 30 miles away from Silicon Valley! –  kilolima Aug 16 '12 at 21:12
    
OK, in that case the bandwidth isn't TOO bad. –  Michael Hampton Aug 16 '12 at 21:18
    
No, not too bad but it makes the use of a cloud backup service impossible! :( But thanks for the heads-up on rdiff and not using archives. Do you think if I run a rdiff-backup of /home to an external drive, and then move and connect that external drive to the remote NAS and copy the rdiff directory to the NAS's drive, the next time rdiff-backup is run from the server over SSH to the NAS it will be fooled into not trying to copy over everything again? –  kilolima Aug 16 '12 at 21:52
    
Only if you leave the backup on the external drive. –  Michael Hampton Aug 16 '12 at 21:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.