Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We have a case of file transfer speed capping (maximum speed) as follows:

  • 1 file transfer capped at 5Mbps;
  • 2 parallel file transfers (same file) capped at 10 Mbps

What can explain such a behavior?

It does not seem to be the IP layer (network). Would that be a setting at TCP layer?


Edit:

I'll try to clarify the context and my question(s).

The file transfer occur between two FTP servers (I am not sure which OS or FTP software), across a Wide Area network (private network, not Internet). So we have something like this:

FTP server A --> LAN --> WAN --> LAN --> FTP server B

The first file transfer job gets 5Mbps bandwidth. We thought the WAN had a bottleneck somewhere which capped the bandwidth. But then the additional file transfer job manages to find another 5Mbps of bandwidth...

share|improve this question
    
curious as to which ftp client you are using? and is it multithreaded? –  micah Mar 1 '10 at 19:49
add comment

5 Answers

If your latency is high (e.g. if you're transferring across the Internet) it could be because one or both ends of the connection don't support window scaling. If your ISP is inclined to mess with your packets it could be due to traffic shaping. As Robert Moir points out, it could be an application or driver issue.

share|improve this answer
    
+1 for traffic shaping possibility (QoS) –  quadruplebucky Mar 1 '10 at 20:18
add comment

The way you stated the question is a little confusing. Let me restate it more explicitly.

On a given network, 1. One file transfer it transmits about 5Mbps max. Nothing you do gets more throughput. 2. Two file transfers, run concurrently, transmit about 10Mbps max. Why can two transmit more than one?

The answer is that TCP/IP is a polite protocol. It actively tries not to hog a channel. It is actually optimized such that many transfers at once all get a relatively fair share of bandwidth.

The down side of that is that a single transfer on an otherwise empty network gets pretty bad performance. This isn't because the creators of TCP/IP want it that way, years of research just haven't improved that situation very much because the primary research is making the internet as a whole more efficient, not making a particular pair of machines on a closed network do well.

It is actually a lot harder to tune a network for your particular situation. For example, when your machine is waiting for the hard disk, the network can go idle, unused. With two transfers at the same time, while one transfer is waiting for the hard disk, another transfer is "keeping the pipe full".

If you want to improve transfer rate break your transfers into multiple simultaneous transfers. Do benchmarks to see if 1, 2, 3, or 4 get the best wall-clock time. Use what works best. (and if you ever change your network, NIC speed, etc. etc. you should re-do the benchmarks)

If you can't break the task into multiple transfers, you'll need to look into tuning the TCP/IP settings on your operating system. There are some parameters you can change, such as how fast it should ramp up bandwidth use ("fast start timer") and other settings. These are different for different operating systems. Searching teh google finds articles like http://support.microsoft.com/kb/93444 and http://www.psc.edu/networking/projects/tcptune/ However I should warn you that you can make things worse just as easily than you can make things better. You should establish a benchmark and tune one parameter at a time running the entire benchmark after each change. Keep good logs and see what makes things better and worse. And also, remember that once you have tuned things for this kind of transfer, doing other networking tasks (like surfing web pages) might be out of tune.

share|improve this answer
    
Are you saying then that it is the TCP settings on the FTP servers which cause this? And why 5Mbps? –  Steven Mar 1 '10 at 19:42
    
This isn't exactly true. TCP in and of itself is not a "polite" protocol and will happily stomp all over itself unless QoS is implemented. QoS = Quality of Service. –  Chris Lively Mar 1 '10 at 19:53
add comment

Impossible to say with the information you're giving here. I've seen file transfers go faster than 5mb for a single download so it isn't some immutable "law of teh interwebs". Could be the application you're using, of course. Could be a setting. Could be a driver.

share|improve this answer
add comment

What protocol, program, local or via internet? There is no reason why an application can cap each request/transfer at 5Mb/s.

share|improve this answer
add comment

Somehwere in the network each TCP stream is getting capped to 5 Mbps. Most likely in the WAN connection, but it could be elsewhere.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.