1

I’m trying to improve my TCP throughput over a “high delay network”, with some packet drops, between Linux machines.

I set tcp_mem, tcp_wmem and tcp_rmem to “8192 20530752 20530752”.
I set rmem_max, wmem_max, rmem_default and wmem_default to “20530752”.
I set netdev_max_backlog and txqueuelen to 50000.
I set tcp_congestion_control to “scalable”.

I’m using “nist” (cnistnet) to simulate a delay of 50ms (25ms for each direction) and drop of 0.5% (0.25% for each direction), and the BW I reach is about 7.48mbps.
Here is a table of the results (using iperf to measure the rates):

     |   0ms    |   50ms
0%   | 710mbps  | 276mbps
0.5% | 181mbps  | 7.48mbps

I didn’t expect the latency to have so much effect on the throughput (not with these big TCP windows). I also didn’t expect the drops to have so much effect. Especially not with the “scalable” algorithm, as it’s “congestion window” is supposed to recovery very quickly from packet drops.

I used tcpdump & sar (part of sysstat) to try to see what’s going on. I didn’t see anything suspicious in sar’s reports. And in tcp dump I see:
“rexmt data pkts” = 88 (out of 8183 sent)
“rexmt data bytes” = 127,424 bytes (out of 44,649,104 bytes sent)
“avg owin” = 135,964 bytes.
“avg retr time” = 53.5 ms.

Is this the best TCP can do? Is there no other algorithm which can achieve better throughput in this scenario? Is my scenario out of the ordinary (I think that 50ms delay with 0.5 drop are normal WAN properties)?

5
  • 0.5% packet loss is not small, it is HUGH. Ridiculously hugh. Non working hardware level of loss.
    – TomTom
    Oct 2, 2011 at 13:20
  • @TomTom However, it is reasonable for flaky RF network connections.
    – sysadmin1138
    Oct 2, 2011 at 13:36
  • Nope, still in the area you call support and get them fix their crap setup.
    – TomTom
    Oct 2, 2011 at 15:28
  • You didn't say if TCP window scaling was enabled (not much point allocating lots of memory for the buffers otherwise) and working end-to-end.
    – symcbean
    Oct 3, 2011 at 11:05
  • Window scaling was enabled and proven working in a different scenario (with 0% packet drop)
    – Gil Barash
    Oct 4, 2011 at 12:42

0

You must log in to answer this question.

Browse other questions tagged .