0

I got a strange GRE perfomance issue.

The setup is shown bellow.

[host1] --- GRE over Eth 1G --- [linux router] --- Eth 10G --- [host2]

Iperf transfer rate host1>host2 reaches 950Mbps. It encreases lineary with several hosts on the left running in parallel, up to several Gbps.

But for the direction host2>host1 iperf transer rate is limited to ~400Mbps both for single and multiple parallel tests.

It is notable that:

  1. the observed tcp window size is under 64K with very low retransmition rate.

  2. plenty of packets emmited by the router with ethertype=0x0011 on 10G link . I suppose it is some type of back pressure.

I suspect that GRE routing stack uses very small internal shared buffer (64K) but I could not find any usefull info on it.

I'll be grateful for any advice how to fix this problem.

I use Centos 7 (kernel 3.10.0-693) on the router.

==

[wk@host2 ~]$ iperf3 -c host1 -t 100
Connecting to host host1, port 5201
[  4] local host1 port 33930 connected to host2 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  79.7 MBytes   668 Mbits/sec   10   62.8 KBytes
[  4]   1.00-2.00   sec  68.9 MBytes   578 Mbits/sec    1   5.46 KBytes
[  4]   2.00-3.00   sec  43.7 MBytes   366 Mbits/sec    5   21.8 KBytes
[  4]   3.00-4.00   sec  49.4 MBytes   414 Mbits/sec    2   28.7 KBytes
[  4]   4.00-5.00   sec  52.6 MBytes   442 Mbits/sec    1   43.7 KBytes


reading from file out.pcap, link-type LINUX_SLL (Linux cooked)
13:08:37.355114 IP host2.33197 > host1.5201: Flags [S], seq 3986495932, win 14600, options [mss 1460,sackOK,TS val 3358395421 ecr 0,nop,wscale 9], length 0
13:08:37.355120 Out 3c:fd:fe:9c:e4:20 ethertype Unknown (0x0011), length 80:
        0x0000:  0000 0800 4500 003c 15a1 4000 4006 9d18  ....E..<..@.@...
13:08:37.355642  In 00:1b:21:bc:5d:96 ethertype Unknown (0x0011), length 80:
        0x0000:  0000 0800 4500 003c 0000 4000 3f06 b3b9  ....E..<..@.?...
13:08:37.355642 IP host1.5201 > host2.33197: Flags [S.], seq 1075451774, ack 3986495933, win 13980, options [mss 1410,sackOK,TS val 3173856449 ecr 3358395421,nop,wscale 8], length 0
13:08:37.355675 IP host2.33197 > host1.5201: Flags [.], ack 1, win 29, options [nop,nop,TS val 3358395421 ecr 3173856449], length 0
13:08:37.355678 Out 3c:fd:fe:9c:e4:20 ethertype Unknown (0x0011), length 72:
        0x0000:  0000 0800 4500 0034 15a2 4000 4006 9d1f  ....E..4..@.@...
        0x0010:  c1e8 0218 c1e8 021a 81ad 1451 ed9d 19bd  ...........Q....
13:08:37.355720 IP host2.33197 > host1.5201: Flags [P.], seq 1:38, ack 1, win 29, options [nop,nop,TS val 3358395421 ecr 3173856449], length 37
13:08:37.355725 Out 3c:fd:fe:9c:e4:20 ethertype Unknown (0x0011), length 109:
        0x0000:  0000 0800 4500 0059 15a3 4000 4006 9cf9  ....E..Y..@.@...
        0x0010:  c1e8 0218 c1e8 021a 81ad 1451 ed9d 19bd  ...........Q....
Improve this question
4
  • Using e.g. OpenVPN instead of this obsolete protocol pretty much solves any performance issue, too. ;)
    – Esa Jokinen
    Jun 9, 2018 at 15:54
  • Actualy GRE gives better performance and much more applicable in my case. I belive something is just missconfigured.
    – Veniamin
    Jun 9, 2018 at 17:28
  • Have you tried to disable various offload features on the interface in question?
    – Peter Zhabin
    Jun 11, 2018 at 18:08
  • @PeterZhabin I found the reason, see bellow, thanks anyway.
    – Veniamin
    Jun 13, 2018 at 12:32

1 Answer 1

Reset to default
1

Sorry, It was my mistake.

Stale traffic control 400Mbps restriction was applied to the output of host2.

[root@host2 ~]# tc class show dev eth2.17
class htb 1:1 root prio 0 rate 400000Kbit ceil 400000Kbit burst 125Kb cburst 125Kb
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .