Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

For some reason when running a single rsync command I am getting 1 MB/sec to 2 MB/sec even when I connecting 2 servers both connected to 1 Gbps ports.

rsync -v --progress -e ssh /backup/mysqldata/mysql-bin.000199 root@XXX.XXX.XXX.XXX:/secondary/mysqldata/mysqldata/mysql-bin.000199

I have over 800 GB of data to transfer split among 500 or so files all starting with:

mysql-bin.000*

I've found that running 25-30 rsync simultaneously from seperate SSH windows gets me upwards of 25 MB/sec but it will take me hours to run these all manually.

Is there anyway to get the 25 MB/sec from a single rsync command?

share|improve this question
    
You might want to investigate setting up an rsync daemon, or using something other then SSH for a large transfer like that (RSH). Are they on your local network. Do you really need the encryption overhead? Maybe mount the filesystem using NFS? –  Zoredache Oct 4 '12 at 7:51
    
What's the latency to the remote host? –  Cakemox Oct 4 '12 at 7:54
    
Latency averages 24ms –  reefine Oct 4 '12 at 8:24

2 Answers 2

up vote 3 down vote accepted

OpenSSH is notorious for having window size limits which are not very friendly for high-latency connections. To determine the minimum window size for your desired throughput (let's ignore other overhead for now):

Window size = throughput * latency

So, if you want 25MB/sec (200,000,000 bits/sec) over a 25ms latency (0.025 seconds):

Window size = 200,000,000 * 0.025 = 5000000.0 bits

Divided by 8, that's roughly a 625KB window. Let's flip it around and look at a 64KB window (the internal cap in many versions of OpenSSH) at your latency:

throughput = 512000 bits / 0.025s latency = 20480000.0 bits/sec = ~2.5MB/sec

That's pretty close to what you're seeing (you may have some other overhead as well.)

So, I would check in the following order:

  • Ensure your OS has its TCP window sizes set sanely and autoscaling enabled.
  • Make sure you are using a version of SSH without a fixed internal window size (I believe this was fixed around OpenSSH 4.7) OR install HPN-SSH, which is patched to address these issues.
  • Try using lighter options for SSH (arcfour, for example)
share|improve this answer

SSH performance can vary drastically according to whether compression is on.

share|improve this answer
1  
could you expand on this a little please ? –  Iain Oct 4 '12 at 9:50
1  
If compression is on CPU time is used in an attempt to save on network bandwidth. If the data is highly compressible this may be a net gain - or with more random-looking data it would be a net loss. Check whether your SSH defaults to compressing and see if you get better results with the other option. –  Peter Oct 4 '12 at 16:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.