1

Something broke on my KVM host (Ubuntu 16.04).

I have some virtual machines configured to use br0 so they appear as separate machines on the LAN with their own IP.

They get their IP from the router, via DHCP. All was well.

But of course it stopped working, and I can't understand why. br0 is still there, but the machines can't ever get a DHCP lease. The host still has connectivity. If I switch the machines to NAT, they get access to the network. but not with br0.

Here is /etc/network/interfaces on the host:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
# auto enp4s0
# iface enp4s0 inet manual

# VMs bridge
auto br0
iface br0 inet dhcp
        bridge_ports enp4s0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

$ ip addr:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
    link/ether 00:1d:09:0b:87:6d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::21d:9ff:fe0b:876d/64 scope link 
       valid_lft forever preferred_lft forever
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:1d:09:0b:87:6d brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.129/24 brd 192.168.1.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::21d:9ff:fe0b:876d/64 scope link 
       valid_lft forever preferred_lft forever
4: br-0e57a29cd0f7: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:2b:9a:39:d6 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 scope global br-0e57a29cd0f7
       valid_lft forever preferred_lft forever
5: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ed:8f:cb:c1 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:edff:fe8f:cbc1/64 scope link 
       valid_lft forever preferred_lft forever
7: veth44c7e80@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 3e:61:3e:be:b2:8e brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::3c61:3eff:febe:b28e/64 scope link 
       valid_lft forever preferred_lft forever
8: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:41:79:0f brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
9: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:41:79:0f brd ff:ff:ff:ff:ff:ff

$ brctl show

bridge name     bridge id               STP enabled     interfaces
br-0e57a29cd0f7         8000.02422b9a39d6       no
br0             8000.001d090b876d       no              enp4s0
docker0         8000.0242ed8fcbc1       no              veth44c7e80
virbr0          8000.52540041790f       yes             virbr0-nic

$ ip route list

default via 192.168.1.1 dev br0 
172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.0.1 
172.18.0.0/16 dev br-0e57a29cd0f7  proto kernel  scope link  src 172.18.0.1 linkdown 
192.168.1.0/24 dev br0  proto kernel  scope link  src 192.168.1.129 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 linkdown 

brctl show with a VM started:

bridge name     bridge id               STP enabled     interfaces
br-0e57a29cd0f7         8000.02422b9a39d6       no
br0             8000.001d090b876d       no              enp4s0
                                                        vnet0
docker0         8000.0242ed8fcbc1       no              veth44c7e80
virbr0          8000.52540041790f       yes             virbr0-nic

$ ip address show dev vnet0

13: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:81:08:d0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe81:8d0/64 scope link 
       valid_lft forever preferred_lft forever

I have no idea what to do at this point. I tried removing and reinstalling kvm + qemu, removing and recreating the bridge, creating a new VM on br0... Absolutely no change.

6
  • You might want to completely verify Layer 2 connectivity before trying to troubleshoot layer 3. arping is a good friend here, and you can use it to try to contact a known MAC address within that VLAN. A working example would be arping -I br0 [known_good_mac_address].
    – Spooler
    Jul 4 '17 at 3:50
  • Thanks. I tried with 52:54:00:81:08:d0 and fe:54:00:81:08:d0 (not sure which to use, 52:xxx is the config on the VM, but it shows as fe:xxx as the MAC on vnet0... Timeout for either MACs. Where do I go from there?
    – user306802
    Jul 4 '17 at 12:16
  • How you done any updates recently? Jul 4 '17 at 14:42
  • No, I just ran sudo ip addr add 192.168.1.130/24 dev enp4s0 and then restarted to undo it.
    – user306802
    Jul 4 '17 at 19:35
  • Can you contact anything on layer 2 from the bridge device directly on the host (that is, perform that same arping command from the hypervisor directly to that known good MAC)? I'm trying to see where you layer 2 connectivity is breaking down, because that seems to be the root of the issue. If you can get out on layer 2 from your host, then we need to look closer at bridge forwarding or similar.
    – Spooler
    Jul 5 '17 at 17:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.