Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm trying to backup a huge tar archive (150GB) from a root server (Debian Wheezy) to my local storage server (Cygwin). Speedtests with the root server give me about 100 - 120 MB/s download rate and the local server has a 100 Mbit/s connection that gives me about 10 - 12 MB/s both constantly.

However when I start pushing the file over SSH/SCP/RSYNC to my local server I only get about 2,5MB/s. The strange thing is: when I simultanously start another SSH transfer the throughput increases to 5MB/s, third one 7,5MB/s to the full capacity of the connection.

So my question: is it possible with the standard OpenSSH software to create a multithreaded TCP stream for example 10 connections to transfer one single file in a single command?

Or do you have any other hints how to efficiently use the connection?

share|improve this question
    
I have one server where a single ssh stream goes up to 80/90 MB/s and an other goes up to 20/25 MB/s, both with gigabit nic. You have some other kind of problem. –  Luigi Nov 18 '13 at 12:12
    
Okay, but why do multiple Connections to the same host give me in total the throughput that is expected? Debugging the SSH Connection with -vvvv did not give me any hints. –  durox Nov 18 '13 at 12:34
    
Try another protocol, like ftp or http. If you have the same problem, triple check your network. –  Eric DANNIELOU Nov 18 '13 at 13:32
    
It's the same with HTTP; not a protocol issue. If I start multiple Downloads from the same file the throughput reaches the expected Level. What further checks would you recommend? –  durox Nov 18 '13 at 14:01
    
maybe some bandwidth limit, u should check limits on the target server, on the clients (try different clients), or switch. Under linux to configure bandwidth limit the command is tc –  Luigi Nov 18 '13 at 14:19
show 1 more comment

2 Answers

When you say 'root server' do you mean a hosted server at a colo facility / data center and your 'local server' is using your office's ISP?

If so, your ISP may be bandwidth limiting per TCP connection, or your local router could be doing it.

I wrote a quick HOWTO on how to test bandwidth with with iperf ages ago - it was written for linux but Cygwin has iperf as well. You probably want to at least know for sure that you're dealing with this issue (and possibly get it fixed).

If you can't fix it, try using lftp as a wrapper to multiplex the transfers.

share|improve this answer
add comment

The behavior you are describing is typical for misconfiguration of the network stacks on either or both ends.

SSH uses TCP to transfer data, and here it seems that the TCP window is limiting the throughput. It would seem that your throughput here is limited by T = min(cwnd, rwnd)/RTT. Then when you add another connection, it would be 2T, 3T, and so on up to a point where the window is no longer limiting, but rather the physical connection itself.

Since you are using Cygwin on the receiver side, I'd start looking at the Linux side. You should check the values on the receiver side too. There are many many TCP tuning guides online(for example this one), so here's a small recap.

You can set the following parameters with sysctl -w, or list them all with sysctl -A

The value of BDP belowshould in your case be set to at least 2*100e6 [bits/s] * RTT [s], let's say your RTT is 10 ms then you'll need 250000 bytes. Note that you will need to use Window Scaling to use windows larger than 64k.

net.core.rmem_max = BDP
net.core.wmem_max = BDP
net.ipv4.tcp_rmem = 4096 87380 BDP
net.ipv4.tcp_wmem = 4096 65536 BDP

sysctl net.ipv4.tcp_window_scaling = 1
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.