I don't know what the issue is here...

Here's my setup.

I have a reasonably fast Desktop PC sitting on three T1's (twisted into a single line? or however they are joined) which normally gives us about 4Mbps up and down. The desktop has a static IP, and we're currently hosting it in midtown Manhattan.

We just built a dedicated server box at a hosting company in Brooklyn, which comprises of a much faster machine sitting on a 10Mbps unmetered and reportedly unthrottled line (has a dedicated IP as well). The server is running Windows Server 2008.

So on one side of the equation I should have 4Mbps up, and on the other I should have 10Mbps down, yet when I transfer a large file (2Gb) I'm getting speeds around 200Kbps!! Wtf?!?

Here is a traceroute from here to there:

traceroute to 66.109.xxx.xxx (66.109.xxx.xxx), 64 hops max, 52 byte packets
 1 (  1.459 ms  0.744 ms  0.670 ms
 2 (  44.353 ms  15.398 ms  11.758 ms
 3  ip-160-79-127-229.autorev.intellispace.net (  14.341 ms  10.849 ms      8.220 ms
 4  gigabitethernet6-24.ar4.nyc1.gblx.net (  7.620 ms  33.029 ms  33.442 ms
 5 (  23.604 ms  31.267 ms  19.221 ms
 6  galaxy-visions.nyc2.webair.net (  6.183 ms  4.622 ms  13.551 ms
 7  *^C

(Step 7 is my server, as the traceroute is blocked by the firewall).

I've ran speedtests on both machines (speedtest.net) and the numbers roughly match what I should be getting (what I listed earlier).

The strange thing is that when I start my transfer of a 2Gb file, the transfer speeds for the first 1Mb are around 1Mbps, then it falls steadily after that until it flattens out at anywhere between 200-300kbps. I don't understand why I'm getting such slow speeds from these two very high speed connections!!

Any thoughts?

  • What tool are you using to transfer your file? – Zoredache Sep 18 '09 at 7:11
  • Also, I suggest you test between bandwidth between the two systems with something like iperf. (iperf.sourceforge.net) – Zoredache Sep 18 '09 at 7:15
  • I'm using scp (through ssh). Both through Cygwin. – neezer Sep 18 '09 at 8:05

This is by no means a complete answer, but what is the MTU on the packets, and does the switching/routing equipment in between support that MTU if it's not of a standard size (for example jumboframes).

Also, can you see if packets are being dropped?

Are there other machines at each end you can do a similar testing with to try and rule out the problem being one of the hosts themselves?

Finally, if you do a speed test between each end point, and a third-party node, do you find anything interesting?

If you do some ping tests (set the no-fragment flag, and try various packet sizes), it may help to have tcpdump/wireshark running - so you can see any messages being sent back from intermediate routers which may tell you why the connection speed slows down to a grind.

  • How would I find the MTU on the packets? I believe our routing equipment is capable of jumboframes (don't know off the top of my head, though). How do I see if packets are being dropped? Other machines on the 4Mbps network have similar speeds uploading to our dedicated box, but I don't have other machines on the Brooklyn side to test with. Not sure what "anything interesting" would entail in your last question; the traceroute and the speed test (@ speedtest.net) don't seem to say anything interesting that I can see. – neezer Sep 18 '09 at 6:51
  • Well you can use ping and set the MTU yourself to begin with, on Windows I think you'd do something like "ping -f -l 1472", and if there are no dropped packets, then go higher. 1472 + 28 overhead makes up the 1500 is the traditional size, but it's a hangover from long ago, and you should be able to go higher than that. The -f you'll need to double check - it's the option that says "don't fragment the packets". – Xerxes Sep 18 '09 at 7:48
  • Regarding "anything interesting" - I'm trying to see out of all the permutations of 2 hosts, if any have a considerably improved throughput than others, say node A is one end, B the other, C a machine in A's subnet, D a machine on B's subnet, and E an external host, if you try various pairs from A-E, do you see some that are faster than others, and that may help in troubleshooting this further. – Xerxes Sep 18 '09 at 7:50
  • (host `E' in the above comment would be speedtest.net) – Xerxes Sep 18 '09 at 7:51
  • Also, what protocol are you using to transfer the file with? Is there another protocol you can use instead? – Xerxes Sep 18 '09 at 8:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.