1

We have a case of file transfer speed capping (maximum speed) as follows:

  • 1 file transfer capped at 5Mbps;
  • 2 parallel file transfers (same file) capped at 10 Mbps

What can explain such a behavior?

It does not seem to be the IP layer (network). Would that be a setting at TCP layer?


Edit:

I'll try to clarify the context and my question(s).

The file transfer occur between two FTP servers (I am not sure which OS or FTP software), across a Wide Area network (private network, not Internet). So we have something like this:

FTP server A --> LAN --> WAN --> LAN --> FTP server B

The first file transfer job gets 5Mbps bandwidth. We thought the WAN had a bottleneck somewhere which capped the bandwidth. But then the additional file transfer job manages to find another 5Mbps of bandwidth...

1
  • curious as to which ftp client you are using? and is it multithreaded?
    – micah
    Mar 1, 2010 at 19:49

4 Answers 4

4

If your latency is high (e.g. if you're transferring across the Internet) it could be because one or both ends of the connection don't support window scaling. If your ISP is inclined to mess with your packets it could be due to traffic shaping. As Robert Moir points out, it could be an application or driver issue.

1
  • +1 for traffic shaping possibility (QoS) Mar 1, 2010 at 20:18
4

The way you stated the question is a little confusing. Let me restate it more explicitly.

On a given network...

  1. One file transfer it transmits about 5Mbps max. Nothing we try do gets more throughput.
  2. Two file transfers, run concurrently, transmit about 10Mbps max (5M + 5M).

There are no box in the middle doing "traffic shaping"... the limit is based on the type of network we have.

Why can two transmit more than one?

The answer is that TCP/IP is a polite protocol. It actively tries not to hog a channel. It is actually optimized such that many transfers at once all get a "fair share" of bandwidth.

The down side of that is that a single transfer on an otherwise empty network gets pretty bad performance. This isn't because the creators of TCP/IP want it that way. Years of research just haven't improved that situation very much because the primary research is making the internet as a whole more efficient, not making a particular pair of machines on a closed network do well. The internet isn't about you, it is about "us".

It is difficult to tune a network for your particular situation. For example, when your machine is waiting for the hard disk, the network can go idle, unused. With two transfers at the same time, while one transfer is waiting for the hard disk, another transfer is "keeping the pipe full".

If you want to improve transfer rate break your transfers into multiple simultaneous transfers. Do benchmarks to see if 1, 2, 3, or 4 get the best wall-clock time. Use what works best. (and if you ever change your network, NIC speed, etc. etc. you should re-do the benchmarks)

If you can't break the task into multiple transfers, you'll need to look into tuning the TCP/IP settings on your operating system. There are some parameters you can change, such as how fast it should ramp up bandwidth use ("fast start timer") and other settings. These are different for different operating systems. Searching the web finds articles like http://support.microsoft.com/kb/93444 and http://www.psc.edu/networking/projects/tcptune/ However I should warn you that you can make things worse just as easily than you can make things better. You should establish a benchmark and tune one parameter at a time running the entire benchmark after each change. Keep good logs and see what makes things better and worse. And also, remember that once you have tuned things for this kind of transfer, doing other networking tasks (like surfing web pages) might be out of tune.

3
  • Are you saying then that it is the TCP settings on the FTP servers which cause this? And why 5Mbps?
    – Steven
    Mar 1, 2010 at 19:42
  • 1
    This isn't exactly true. TCP in and of itself is not a "polite" protocol and will happily stomp all over itself unless QoS is implemented. QoS = Quality of Service.
    – NotMe
    Mar 1, 2010 at 19:53
  • @notme I respectfully disagree. Here are a list of TCP revisions that have been made to make TCP polite i.e. play well with others. en.wikipedia.org/wiki/TCP_congestion_control In fact, most of that research is due to the fact that is backs off too much. You may be thinking of "UDP" which will stomp all over a network.
    – TomOnTime
    May 23, 2016 at 14:48
1

Impossible to say with the information you're giving here. I've seen file transfers go faster than 5mb for a single download so it isn't some immutable "law of teh interwebs". Could be the application you're using, of course. Could be a setting. Could be a driver.

0

Somehwere in the network each TCP stream is getting capped to 5 Mbps. Most likely in the WAN connection, but it could be elsewhere.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .