I have a mirroring system in place with rsync and it works well. However, moving a file causes it to be transferred again in its entirety.

Given what I know about the rsync algorithm's rolling checksum, wouldn't it be possible to:

  1. tar all the source files
  2. rsync the tar

and only incur a block-size penalty for the change in file name, after which the checksum would resynchronise? However, i don't have the temporary spool space for a large .tar file.

My questions:

Is there an efficient, read/write way to mount a filesystem as a tar, which i can rsync?

Is there a way of piping tar | rsync in some way to achieve this behaviour?


If you use gzip instead of tar, there is the rsyncable option.

Gzip the source, then use rsync to copy it to the destination. Each night, update the gzip archive, and use rsync again. rsyncable will update the gzip archive in such a way that it will comply with rsync's update algorithm and only the changed parts within the gzip archive are sync'ed across each night, not the whole archive.

  • Thanks for your suggestion - but as per the question i don't have the spool space for a temporary file (the volume to sync is multiple terabytes). As of this comment i still don't have a good solution (but i'm now considering rsync'ing the block device directly) – mappu Jun 12 '13 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.