We're looking to backup about 100gb+ of data containing small files (10kb+) each. The backup needs to be done as fast as possible to another harddrive weekly. Which is the better (especially speed wise) way to backup in such scenario? Rsync, or tar?
|
feedback
|
|
rsync can be somewhat painful if you have a very large number of files - especially if your rsync version is lower than 3. On the other hand: if you use tar, you would generate a very big resulting tar-file (unless the data may be compressed a lot). Personally, I would look at rdiff-backup, but make sure that you test your restore situation: rdiff-backup can be very memory demanding when restoring. | |||
|
feedback
|
|
Definitely The advantage of rsync is that it will copy only the files which have changed. If you have 100GB+ of relatively small files, you don't want to copy them all each time. Note: the first backup with Be sure to familiarise yourself with all the options of Tar is an archive utility. You could conceivably create a tar file for the entire 100GB+, but you don't want to transfer it all, each time. | |||
|
feedback
|
|
if your files do not change much - i would vote for rsync. | |||
|
feedback
|
|
Do you need history (multiple backups) or just a plain copy of your data to some other disk? Backing up 100GB of 10KB files would take ages if you don't use a block level backup. Think about making block level snapshots or some other block level based solution, if you really need a fast solution. | |||||||||||||||
feedback
|
|
Take a look at rsnapshot, it's just a script that you can use as front for rsync. It will only backup stuff that has changed and rotate your backups. | |||
|
feedback
|
