I have two dedicated servers hosted by OVH, an SP-128 and an MG-512. The hardware is similar (they are both Supermicro-based builds) but the SP-128 has a Xeon E5 Ivy Bridge and 1 Gbps Ethernet; the MG-512 has dual Xeon E5 Haswell and 10 Gbps Ethernet.
I'm in the process of migrating everything from the old server (the SP-128) to the new server (the MG-512). Here's what I've done so far:
- Set up the partitions on the new box
- Copied the data and OS (the entire ZFS storage pool, all-inclusive) from the old to the new box using
zfs send | ... ssh "zfs recv"
- Reinstalled GRUB
- The new server has a new public IP, so I changed that in
/etc/network/interfaces, and it has a new "name" for its primary Ethernet adapter, so I updated the config (old:
- I also updated the libvirt network adapter,
macvtap-netin libvirt) replacing
enp3s0f0in the configuration, and have done a
virsh destroy macvtap-net; virsh start macvtap-netto ensure the change took effect.
- The new server boots and is accessible over the network!
Now, I have a bunch of LXD and libvirt/KVM guests. Each of them has one or more static IPs that they claim from one of two public /27 subnets I have. OVH has migrated both /27s over to my new server (at my request). Here are some more details of the networking setup:
- The LXD containers are connected using macvlan. The networking works. All of my LXD containers can reach (and be reached by) the public Internet.
- The KVM guests are connected using macvtap. The networking doesn't work.
- The default gateway and the subnet used by the LXD and the KVM guests is exactly the same; the only thing that differs is the MAC address used by each guest/container, and the specific IP within the /27. I haven't changed the guest's IP configuration because it shouldn't need to be changed.
My problem is the above: The networking for the KVM guests doesn't work. By "doesn't work", I mean the guests are able to set the static IP as before, but they are completely unreachable. The guests can't ping their default gateway or any other IP, public or private.
I have the old and new server running simultaneously, so I've fairly thoroughly checked that the configuration is the same between them. For instance, the old server does not have promiscuous mode enabled for the primary ethernet adapter, nor for macvtap0. Neither does the new server.
Most other configuration stuff couldn't have changed, because the files that make up the configuration of the OS are bit for bit copied from the old server to the new. The libvirt/KVM configuration, for example, was not modified or recreated between the servers; it is literally the same configuration copied over as part of the filesystem-level data transfer with
zfs send / zfs recv.
So, the way I see it, the variables at play are:
- The "main" public IPv4 address of the physical box itself changed from the old to the new server.
- The model of the Ethernet adapter changed, from an Intel Gigabit Ethernet to 10G.
- The name of the Ethernet adapter changed, from
- The default gateway for the physical box changed, but the default gateway for the /27s is still routable and usable (it works in LXD with macvlan).
I've triple-checked the MAC addresses of the KVM virtio-net adapters are correct, because OVH performs MAC address filtering (mainly to prevent accidental misconfiguration, not as a security measure) and has assigned a specific MAC address to each IP I'm using.
So I'm at a loss. Why can't my KVM guests (I have one Linux and one Windows) access the network on the new box, when basically everything is the same, and the only things that were legitimate changes have already been changed?
Oh, and I also changed the network adapter in each of the libvirt guest's config (and rebooted the guests). Here's what that looks like now:
<interface type='direct'> <mac address='re:da:ct:ed'/> <source dev='enp3s0f0' mode='bridge'/> <target dev='macvtap0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface>
enp3s0f0 is in there too (used to be
Here is some additional troubleshooting stuff:
ip -d link macvtap0 show on the new box:
18: macvtap0@enp3s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 500 link/ether re:da:ct:ed brd ff:ff:ff:ff:ff:ff promiscuity 0 macvtap mode bridge addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
On the old box, the output is 100% identical except for replace