I’m trying to improve my TCP throughput over a “high delay network”, with some packet drops, between Linux machines.
I set tcp_mem
, tcp_wmem
and tcp_rmem
to “8192 20530752 20530752”.
I set rmem_max
, wmem_max
, rmem_default
and wmem_default
to “20530752”.
I set netdev_max_backlog
and txqueuelen
to 50000.
I set tcp_congestion_control
to “scalable”.
I’m using “nist” (cnistnet) to simulate a delay of 50ms (25ms for each direction) and drop of 0.5% (0.25% for each direction), and the BW I reach is about 7.48mbps.
Here is a table of the results (using iperf to measure the rates):
| 0ms | 50ms
0% | 710mbps | 276mbps
0.5% | 181mbps | 7.48mbps
I didn’t expect the latency to have so much effect on the throughput (not with these big TCP windows). I also didn’t expect the drops to have so much effect. Especially not with the “scalable” algorithm, as it’s “congestion window” is supposed to recovery very quickly from packet drops.
I used tcpdump
& sar
(part of sysstat) to try to see what’s going on. I didn’t see anything suspicious in sar’s reports. And in tcp dump I see:
“rexmt data pkts” = 88 (out of 8183 sent)
“rexmt data bytes” = 127,424 bytes (out of 44,649,104 bytes sent)
“avg owin” = 135,964 bytes.
“avg retr time” = 53.5 ms.
Is this the best TCP can do? Is there no other algorithm which can achieve better throughput in this scenario? Is my scenario out of the ordinary (I think that 50ms delay with 0.5 drop are normal WAN properties)?