Site A: Denver datacenter. 60MBPS.
Site B: Chicago. 100MBPS.

ICMP pings:
Packets: Sent = 176, Received = 176, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 74ms, Maximum = 94ms, Average = 75ms

File transfer between sites that never goes past ~7MBPS:


Windows Update download at 60MBPS+:


Site to site: IPSec VPN using two Cisco 5520's. CPU at 3-4% and lots of memory to spare.

The latency between to two sites is very acceptable so I can't see an issue why it is performing so slow when transferring between the two sites.

I have found that any type of transfer (FTP, HTTP, Windows file shares) will never go above ~7MBPS.

When the WAN was first setup, I was able to get transfers at 50-60MBPS, which is what is expected due to the WAN connection at the Site A at 60MBPS. Then a few days later, I was not able to get anything going faster than ~7MBPS.

Is there a upstream router between Denver and Chicago causing this? I want to take the blame away from our setup as downloads from Windows Update go blazing fast and for the first few days after the site to site VPN came up, I was transferring VM images at 50-60MBPS.

Our stack: HP P2000 MSA -> HP C7000 Chassis -> HP Flex-10 -> Cisco Gigabit switch -> Cisco ASA -> WAN

  • 1
    Is TCP window scaling enabled on your servers? Have you configured the ASA not to clobber window scaling (tcp-options window-scale allow)? Jan 21 '11 at 22:10
  • +1 regarding TCP window scaling. And are you transferring inside the VPN or directly on the internet?
    – petrus
    Aug 12 '12 at 20:57

IPERF is a very useful tool for testing throughput in situations like this.

If different applications are getting different throughputs, it could be an issue with the application/server used, or that someone has implemented QoS (Quality of Service) on the link, so that the network limits traffic for some applications.

  • Going back to my comment to user48838's answer, how would QOS cause the hosts to adjust their window size, unless the hosts themselves are implementing QOS? Would QOS on any intermediate link cause either host to adjust it's window size? Asking because I honestly don't know.
    – joeqwerty
    Oct 5 '10 at 0:44
  • When packets start being dropped to meet QoS settings, the hosts will react by reducing their window sizes until packets are flowing normally again, then the window sizes should creep up again: Oct 5 '10 at 1:44
  • Specifically, it would be helpful to use iperf to test throughput between the two VPN endpoints, without actually passing traffic through the VPN.
    – Skyhawk
    Nov 18 '12 at 3:49

The upstream throughput on one of the sides may be getting throttled or there is some intermediate equipment/network throttling down the connection between the two points. One possible way to verify and maybe identify the situation is by utilizing a 3rd reference. In this case some of the speed test sites may be worth trying.

  • I admit I'm a little lost. How would intermediate throttling cause one host to adjust it's window size? My understanding has always been that window scaling is a function of the hosts involved in the connection, not the intermediate devices. If the intermediate devices were throttling the data transfer it seems that both hosts would still be advertising their maximum window size, but would actually only be getting as much data as the intermediate devices were passing. A shrinking window size seems like a host problem... buffers getting too full, etc.
    – joeqwerty
    Oct 4 '10 at 23:49
  • A long as there is no truncation of mismatched window sizes, then transfers should not be inhibited - the transfers may not be at maximum efficiency with the additional windowing/framing overhead, but the transfers will still make it across. Throttling may trigger incorrect windowing adjustments at the far ends depending on how they compute/trigger adjustments for auto-tuning implementations (e.g. Vista and Windows 7 TCP/IP stacks - now to include Windows 2008R2).
    – user48838
    Oct 5 '10 at 1:30

Comparing site to site data transfers against a Windows Update download from one site isn't exactly comparing apples to apples if you ask me.

nicelink and nms-dpnss are hosts at each site, correct? cspuni is a host at one site connecting to the Windows Update web site, right? How are those two comparisons related? How is comparing a download between one host and an external web site against a file transfer between two hosts at each site helping you analyze the problem?

My suggestion would be to test data transfers between multiple sets of hosts at each site and compare the results. Example:

A<->B; C<->D; A<->D; B<->C

Where A and C are at one site and B and D are at the other site.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.