22

This is what I need:

  • Rsync daemon running on source server. Destination server does a rsync pull as below:
    rsync -ptv rsync://source_ip:document/source_path/*.abc destination_path/
  • Huge binary files (3 GB to 5 GB) are copied from source machine to destination over a LAN. So no encryption/decryption required. Hence not using SSH. Destination machine directly connects to rsync daemon.
  • The destination folder will always be empty before rsync pull. So I don't have to worry about conditionally updating some files. Rsync pull will always create new files in destination (not update). Hence I don't need rsync to calculate checksums.

I am not satisfied with the amount of CPU consumed by rsync at the receiving end? What rsync options will help me optimize rsync usage in above explained case. What I am concerned about is that rsync might be doing more than what I need. rsync checksums files. I don't need that.

Also would like to see speed improvements.

Changing block-size, disable checksum etc will help?

4
  • 2
    Don't use rsync if there are no deltas to transfer.
    – womble
    May 7, 2012 at 13:01
  • 3
    @womble: What tool should I use then? Rsync was suggested here- serverfault.com/questions/384467/…
    – Varun
    May 7, 2012 at 13:36
  • netcat, or scp with the null cipher.
    – womble
    May 7, 2012 at 13:38
  • 3
    Scp with null cipher? Are you sure scp supports null cipher. Can you please give one example or point me to relevant resources?
    – Varun
    May 9, 2012 at 12:17

1 Answer 1

36

It seems like you might want to take a look at the --whole-file or -W switch. This is enabled by default if rsync is doing local filesystem copies, but I think the docs recommend its usage if the LAN speed is high and syncing over the network.

Basically, it disables the rsync delta algorithm and just transfers the whole file if it believes it's different.

You shouldn't have to worry about checksumming unless you force -c. By default, it just looks at the last modified time and the file size, and assumes the files are different if those values are different.

1
  • What are the advantages and disadvantages of --whole-file? Thanks
    – xralf
    Nov 1, 2022 at 17:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .