Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I've tested several cases for cwRsync (3.0) on Win Server 2003. My scenario is have a rsync daemon on a source host with source files ~3GB total. And there're 50 dest servers running rsync command to sync file from source server at the same time. All servers are under same subnet and having gigabyte network adapter.

Test case 1: rsync -av This will do delta transfer, but the performance is really bad.

Number of files: 160
Number of files transferred: 49
Total file size: 2993222827 bytes = 2854 MB
Total transferred file size: 1847285024 bytes =1761 MB
Literal data: 69543644 bytes = 66 MB
Matched data: 1777741380 bytes = 1695 MB
File list size: 3088
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 1256178
Total bytes received: 70240527 = 67 MB

sent 1256178 bytes  received 70240527 bytes  182157.21 bytes/sec =

0.17 MB/s total size is 2993222827 speedup is 41.87

time elapsed: 385 sec

Test case 2: rsync -avW Consuming more BW, ends up getting better performance.

Number of files: 160 Number of files transferred: 49 Total file size:
2993222827 bytes = 2855 MB Total
transferred file size: 1847285024
bytes = 1762 MB Literal data:
1847285024 bytes = 1762 MB Matched
data: 0 bytes File list size: 3088
File list generation time: 0.032
seconds File list transfer time: 0.000
seconds Total bytes sent: 1306 Total
bytes received: 1847516035

sent 1306 bytes  received 1847516035 bytes  8085415.06 bytes/sec
7.71 MB/s total size is 2993222827  speedup is
1.62

time elapsed: 228 sec

With this result. I may even want to zip up 3GB files into a 1GB zip file, and rsync smaller file to dest servers. Then unzip on dest servers.

Can the community give some suggestions on how to get the best performance from cwRsync? It seems like it doesn't take any advantage on delta-transfer or incremental copy at all.

share|improve this question

1 Answer 1

If these servers are all on the same LAN, then it might not be worth the overhead that rsync incurs (hence the speedup when you use -W). Using -W is often better when the network I/O is close, or greater than the disk I/O speed, so this isn't too surprising.

What speed does a normal cp run at?

You may also want to look into batch mode, as it will only process the file deltas once, since it knows that all the files on the destinations are identical.

share|improve this answer
    
a normal cp is also close to 7-8MB/s –  Stan Oct 29 '11 at 15:50

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.