For some reason when running a single rsync command I am getting 1 MB/sec to 2 MB/sec even when I connecting 2 servers both connected to 1 Gbps ports.

rsync -v --progress -e ssh /backup/mysqldata/mysql-bin.000199 root@XXX.XXX.XXX.XXX:/secondary/mysqldata/mysqldata/mysql-bin.000199

I have over 800 GB of data to transfer split among 500 or so files all starting with:


I've found that running 25-30 rsync simultaneously from seperate SSH windows gets me upwards of 25 MB/sec but it will take me hours to run these all manually.

Is there anyway to get the 25 MB/sec from a single rsync command?

  • You might want to investigate setting up an rsync daemon, or using something other then SSH for a large transfer like that (RSH). Are they on your local network. Do you really need the encryption overhead? Maybe mount the filesystem using NFS? – Zoredache Oct 4 '12 at 7:51
  • What's the latency to the remote host? – Cakemox Oct 4 '12 at 7:54
  • Latency averages 24ms – reefine Oct 4 '12 at 8:24

OpenSSH is notorious for having window size limits which are not very friendly for high-latency connections. To determine the minimum window size for your desired throughput (let's ignore other overhead for now):

Window size = throughput * latency

So, if you want 25MB/sec (200,000,000 bits/sec) over a 25ms latency (0.025 seconds):

Window size = 200,000,000 * 0.025 = 5000000.0 bits

Divided by 8, that's roughly a 625KB window. Let's flip it around and look at a 64KB window (the internal cap in many versions of OpenSSH) at your latency:

throughput = 512000 bits / 0.025s latency = 20480000.0 bits/sec = ~2.5MB/sec

That's pretty close to what you're seeing (you may have some other overhead as well.)

So, I would check in the following order:

  • Ensure your OS has its TCP window sizes set sanely and autoscaling enabled.
  • Make sure you are using a version of SSH without a fixed internal window size (I believe this was fixed around OpenSSH 4.7) OR install HPN-SSH, which is patched to address these issues.
  • Try using lighter options for SSH (arcfour, for example)

SSH performance can vary drastically according to whether compression is on.

  • 1
    could you expand on this a little please ? – Iain Oct 4 '12 at 9:50
  • 1
    If compression is on CPU time is used in an attempt to save on network bandwidth. If the data is highly compressible this may be a net gain - or with more random-looking data it would be a net loss. Check whether your SSH defaults to compressing and see if you get better results with the other option. – Peter Oct 4 '12 at 16:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.