I have a KVM virtualization server setup at Hetzner. Hetzner provides me with a master IP (95.xxx.xxx.235) and a /29 IPv4 subnet (95.xxx.xxx.184/29) and a /64 IPv6 network (2a01:xxxx:xxxx:xxxx::/64).
The KVM Guest (Debian Stretch) loses IPv6 connectivity exactly after 20 minutes of either reboot on networking services restart. Even though the connection is lost, I can ping the default gateway (fe80::1). The IPv4 connectivity stays up all the time and has no issues.
Currently the interface is set as a macvlan in bridge mode, and I've also tried VEPA and private modes with no luck. Also the NIC type is set to e1000, but I've also tried virtio with no luck.
After connection loss I've taken a TCP dump from the physical NIC on the host, and it shows that there is echo requests leaving and echo replies arriving the interface, but a tcpdump taken from the guests NIC I could see only the requests leaving the NIC.
/etc/network/interfaces on Host:
auto lo iface lo inet loopback iface lo inet6 loopback auto enp2s0 iface enp2s0 inet static address 95.xxx.xxx.235 netmask 255.255.255.192 gateway 95.xxx.xxx.193 up route add -net 95.xxx.xxx.192 netmask 255.255.255.192 gw 95.xxx.xxx.193 dev enp2s0 iface enp2s0 inet6 static address 2a01:xxx:xxx:xxx::2 netmask 64 gateway fe80::1
/etc/network/interfaces on Guest:
auto lo iface lo inet loopback iface lo inet6 loopback auto ens3 iface ens3 inet static address 95.xxx.xxx.187 netmask 255.255.255.248 gateway 95.xxx.xxx.185 iface ens3 inet6 static address 2a01:xxx:xxx:xxx::20 netmask 64 gateway fe80::1
# route -6 -n on Host:
Kernel IPv6 routing table Destination Next Hop Flag Met Ref Use If 2a01:xxxx:xxxx:xxxx::/64 :: U 256 8 1162 enp2s0 fe80::/64 :: U 256 0 0 macvtap0 fe80::/64 :: U 256 0 0 enp2s0 ::/0 fe80::1 UG 1024 8 4534 enp2s0 ::/0 :: !n -1 1 11069 lo ::1/128 :: Un 0 9 81 lo 2a01:xxxx:xxxx:xxxx::/128 :: Un 0 1 0 lo 2a01:xxxx:xxxx:xxxx::2/128 :: Un 0 9 82 lo fe80::/128 :: Un 0 1 0 lo fe80::/128 :: Un 0 1 0 lo fe80::/128 :: Un 0 1 0 lo fe80::xxxx:xxxx:xxxx:1069/128 :: Un 0 1 0 lo fe80::xxxx:xxxx:xxxx:22e1/128 :: Un 0 1 0 lo fe80::xxxx:xxxx:xxxx:201/128 :: Un 0 2 79 lo ff00::/8 :: U 256 0 0 macvtap0 ff00::/8 :: U 256 0 0 enp2s0 ::/0 :: !n -1 1 11069 lo
# route -6 -n on Guest:
Kernel IPv6 routing table Destination Next Hop Flag Met Ref Use If 2a01:xxxx:xxxx:1414::/64 :: U 256 0 0 ens3 fe80::/64 :: U 256 0 0 ens3 ::/0 fe80::1 UG 1024 2 77 ens3 ::/0 :: !n -1 1 6846 lo ::1/128 :: Un 0 5 525 lo 2a01:xxxx:xxxx:xxx::20/128 :: Un 0 3 70 lo fe80::xxxx:xxxx:xxx:22e1/128 :: Un 0 2 6 lo ff00::/8 :: U 256 0 0 ens3 ::/0 :: !n -1 1 6846 lo
# ip -6 neigh on Host:
2a01:xxxx:xxxx:xxxx::20 dev enp2s0 FAILED fe80::1 dev enp2s0 lladdr xx:xx:xx:8d:22:06 router STALE
# ip -6 neigh on Guest:
fe80::1 dev enp2s0 lladdr xx:xx:xx:8d:22:06 router REACHABLE
Probably relevant things from /etc/sysctl.conf on Host:
net.ipv4.ip_forward=1 net.ipv4.conf.enp2s0.send_redirects=0 net.ipv6.conf.all.forwarding=1
Probably relevant things from /etc/sysctl.conf on Guest:
net.ipv6.conf.default.accept_ra=2 net.ipv6.conf.default.autoconf=1 net.ipv6.conf.all.accept_ra=2 net.ipv6.conf.all.autoconf=1 net.ipv6.conf.ens3.accept_ra=2 net.ipv6.conf.ens3.autoconf=1
Probably relevant section of the Guest libvirt-config:
<interface type='direct' trustGuestRxFilters='yes'> <mac address='xx:xx:xx:xx:xx:xx'/> <source dev='enp2s0' mode='bridge'/> <model type='e1000'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface>
As I've been struggling with this for about two weeks and read almost every relevant post with similar issues I've seen that Hetzner apparently does some kind of fishy IPv6 implementations. I've already contacted them, but they wondered that I would have a routing problem myself. That could be true, since after 20 minutes I still receive the echo replies on the physical NIC, even though they aren't forwarded to Guest.
So, any ideas from fellow IPv6'ers?
So Hetzner confirmed me, that 2a01:xxxx:xxxx:xxxx::/64 network is routed to the link-local address of the physical interface. When restarting networking, the NDP entry stays in for 20 minutes but does get removed afterwards as the VM does not answer with correct link-local address, as it has different MAC-address.
It is starting to seem like that I cannot use macvtap-interfaces here, but I have to create a bridge for this. However, I wonder why the host cannot see the guest (and vice versa) with IPv6, when the IPv4 still works. I think it would allow me to route traffic straight from the main link-local address.