3

i'm observing rather strange (at least for me) behaviour for few routes in my network. i use gre interfaces on top of ipsec tunnels. these gre interfaces have mtu 1400 on my entire network. normally i see this kind of tracepath output:

1?: [LOCALHOST]                                         pmtu 1500
 1:  10.xxx.101.1                                         13.625ms
 1:  10.xxx.101.1                                         13.178ms
 2:  10.xxx.101.1                                         13.973ms pmtu 1400
 2:  192.168.yyy.251                                      56.555ms
 3:  192.168.yyy.92                                      643.252ms 
 4:  192.168.yyy.28                                      417.291ms 
 5:  192.168.zzz.129                                     517.893ms reached

but for some reason i got one case when tracepath gives different result:

 1?: [LOCALHOST]                                         pmtu 1500
 1:  10.xxx.101.1                                         13.625ms
 1:  10.xxx.101.1                                         20.857ms
 2:  10.xxx.101.1                                         11.954ms pmtu 1400
 2:  192.168.yyy.251                                      46.456ms
 3:  192.168.yyy.251                                      45.563ms pmtu 1376
 3:  10.zzz.251.1                                         56.648ms
 4:  10.zzz.255.111                                       55.212ms reached

all gre interfaces on 192.168.yyy.251 have mtu 1400, all are configured identically. the 10.zzz.251.1 router doesn't send any ICMP fragm needed packets, gre interface of course with mtu 1400. 192.168.yyy.251 generates ICMP fragm needed but i have no clue why. ip route get 10.zzz.255.111 on 192.168.yyy.251 router shows:

10.zzz.255.111 from 10.xxx.101.253 tos lowdelay via 10.zzz.251.1 dev GRE_OUTPUT_INTERFACE  src 192.168.yyy.251  mark 0x2071
    cache  expires 264sec ipid 0x607b mtu 1376 iif GRE_INPUT_INTERFACE

additionally from time to time (i suspect it depends on traffic) mtu of the route changes and gets heavily lowered and for 10mins (mtu cache expiry) i can't make any new connections that need bigger packets -- but old established connections are working fine. hmmm?

Improve this question

1 Answer 1

Reset to default
0

I would say that this is caused by a GRE tunnel being routed over another GRE tunnel. Your temporary problem may be when an IPSEC tunnel goes down for a few seconds.

Improve this answer
7
  • ipsec connection is rock solid (it's heavily monitored). the 24B difference could indicate that there is something related with GRE but i see no packet from the outside regarding MTU problems. it looks like lowering MTU is done inside 192.168.yyy.251 router. the question is: why? the router has got plenty (over 50) other tunnels that behaves normally (MTU stays at 1400B level)
    – RMielnik
    Jan 4, 2016 at 11:22
  • It must have to do with the route traveled. How are you routing your packets (static, ipsec)? Follow the tunnel and at each hop check the mtu of the interface it's going over. Also check the other way.
    – Law29
    Jan 4, 2016 at 12:39
  • there is a lot of stuff: GRE, IPsec and even dynamic routing. but it's irrelevant i think. i was trying to look into kernel sources and all i saw regarding mtu calculation was mtu of outgoing interface. the router 192.168.yyy.251 is the source of ICMP fragm needed packets, so it's somehow convinced about it but i cannot see why. imho it should not interfere in traffic and just pass 1400B packets because there is no sign of interface with lower mtu anywhere in the path
    – RMielnik
    Jan 4, 2016 at 16:03
  • On 192.168.yyy.251 you have 10.zzz.255.111 from 10.xxx.101.253 tos lowdelay via 10.zzz.251.1 dev GRE_OUTPUT_INTERFACE src 192.168.yyy.251 mark 0x2071 cache expires 264sec ipid 0x607b mtu 1376 iif GRE_INPUT_INTERFACE. What is GRE_OUTPUT_INTERFACE? It is a GRE tunnel, certainly, but is it routed over a real 1500-MTU interface?
    – Law29
    Jan 4, 2016 at 17:41
  • 100% real hardware 1500 MTU interface:) with 50+ other GRE tunnels working like a charm
    – RMielnik
    Jan 5, 2016 at 9:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .