Bridge on vlan on teaming for KVM

Question

I have problem to create bridge on vlan on teaming. Why I do something like that? - Kvm needs bridge - Vlans because I need vm’s in different subnets - Teaming in option active-backup (small HA)

I do it on centos 7 x64, with NetworkManager. Two physical interfaces:

TYPE=Ethernet
DEFROUTE=no
PEERDNS=no
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_PEERDNS=no
IPV6_PEERROUTES=no
IPV6_FAILURE_FATAL=no
NAME=enp8s0f0
UUID=311d4881-f3f1-45af-b01f-b48872b00b0f
ONBOOT=yes

TYPE=Ethernet
DEFROUTE=no
PEERDNS=no
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_PEERDNS=no
IPV6_PEERROUTES=no
IPV6_FAILURE_FATAL=no
NAME=enp8s0f1
UUID=eb43a315-b9e7-42fb-ab52-97e09a162825
ONBOOT=yes

My steps:

nmcli con add con-name team0 ifname team0 type team config '{"runner": {"name": "activebackup"}}'
nmcli con add con-name team0-port1 ifname enp8s0f0 type team-slave master team0
nmcli con add con-name team0-port2 ifname enp8s0f1 type team-slave master team0
nmcli con up team0-port2
nmcli con up team0-port1

After that I have:

NAME         UUID                                  TYPE            DEVICE
virbr0       f8b30bd2-0d92-43af-9b3d-f68c63d6822b  bridge          virbr0
team0        19542d40-dce1-4328-9b64-f44f2fd7f1fb  team            team0
team0-port2  301b2003-18d0-4186-992a-fbc758bb1c7a  802-3-ethernet  enp8s0f1
team0-port1  cafb8b30-60fc-475f-890d-0dc1191bc583  802-3-ethernet  enp8s0f0

When I assign IP to team0, I can ping from this interface gateway. Next I create vlan and bridge:

nmcli con add con-name vlan24-team0 type vlan id 24 dev team0
nmcli con add con-name br24 type bridge ifname vibr24 stp yes
nmcli con add con-name br24-vlan24 type bridge-slave ifname team0.24 master vibr24

nmcli con show:

NAME          UUID                                  TYPE            DEVICE
virbr0        f8b30bd2-0d92-43af-9b3d-f68c63d6822b  bridge          virbr0
team0         19542d40-dce1-4328-9b64-f44f2fd7f1fb  team            team0
enp8s0f0      311d4881-f3f1-45af-b01f-b48872b00b0f  802-3-ethernet  --
enp8s0f1      eb43a315-b9e7-42fb-ab52-97e09a162825  802-3-ethernet  --
br24-vlan24   16fd96b7-10b0-42b6-88e6-38749b33f816  802-3-ethernet  --
br24          667cc267-3726-42cb-a5aa-0a2011167021  bridge          vibr24
vlan24-team0  8ec83969-254b-4f75-8bad-983bac7e1af0  vlan            team0.24
team0-port2   301b2003-18d0-4186-992a-fbc758bb1c7a  802-3-ethernet  enp8s0f1
team0-port1   cafb8b30-60fc-475f-890d-0dc1191bc583  802-3-ethernet  enp8s0f0

ip a:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
6: enp8s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master team0 state UP qlen 1000
    link/ether 00:1b:21:79:55:4f brd ff:ff:ff:ff:ff:ff
7: enp8s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master team0 state UP qlen 1000
    link/ether 00:1b:21:79:55:4f brd ff:ff:ff:ff:ff:ff
8: team0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 00:1b:21:79:55:4f brd ff:ff:ff:ff:ff:ff
    inet6 fe80::21b:21ff:fe79:554f/64 scope link
       valid_lft forever preferred_lft forever
9: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
    link/ether 66:0e:c8:c8:9c:37 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
10: team0.24@team0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 00:1b:21:79:55:4f brd ff:ff:ff:ff:ff:ff
    inet6 fe80::21b:21ff:fe79:554f/64 scope link
       valid_lft forever preferred_lft forever
11: vibr24: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
    link/ether ca:88:56:46:e5:18 brd ff:ff:ff:ff:ff:ff

nmcli con up br24-vlan24
Error: no device found for connection 'br24-vlan24'.

nmcli con up br24
Error: Device 'vibr24' is waiting for slaves before proceeding with activation.

teaming works fine, and I think there is problem to map bridge to vlan because interface vibr24 is down, and br24-vlan24 doesn’t map to device vlan24-team0. But I do it first time and maybe I miss something

VLAN=yes
TYPE=Vlan
PHYSDEV=team0
VLAN_ID=24
REORDER_HDR=0
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=vlan24-team0
UUID=8ec83969-254b-4f75-8bad-983bac7e1af0
ONBOOT=yes

TYPE=Ethernet
NAME=br24-vlan24
UUID=16fd96b7-10b0-42b6-88e6-38749b33f816
DEVICE=team0.24
ONBOOT=yes
BRIDGE=vibr24

DEVICE=vibr24
STP=yes
BRIDGING_OPTS=priority=32768
TYPE=Bridge
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=br24
UUID=667cc267-3726-42cb-a5aa-0a2011167021
ONBOOT=yes

On the other site of link i have cisco switch and configured port-channel with trunk. This are all steps which i made, anyone can help me with that?

Answer 1

I am not familiar with NetworkManager. Personally I prefer "old-fashioned" direct approach with iproute2/brctl/vconfig tools. But I've found this bug report & errata: https://bugzilla.redhat.com/show_bug.cgi?id=1183420 which are very related to your problem. You are just having issue with adding VLAN to a bridge instead of bond interface like there. But a workaround from that bug report will work too.

Instead of adding br24-vlan24 (which will fail because you already have team0.24 device associated with vlan24-team connection!), you must edit vlan24-team0 properties to make it slave to virb24.

I've tried to reproduce a configuration similar to yours. But without teaming, because seems like you don't have issues with that part as you can ping from team0. I've created a bridge br24 and vlan id 24 on my NIC enp2s0. The link above suggests then to modify bonding (VLAN in our case) connection like like this:

# nmcli c mod bond-bond0 connection.master br0
# nmcli c mod bond-bond0 connection.slave-type bridge

But this failed for me on Linux Mint with following error:

$ nmcli c mod vlan24-enp2s0 connection.master vibr24
    Error: Failed to modify connection 'vlan24-enp2s0': connection.slave-type: Cannot set 'master' without 'slave-type'
$ nmcli c mod vlan24-enp2s0 connection.slave-type bridge
    Error: Failed to modify connection 'vlan24-enp2s0': connection.master: Slave connections need a valid 'master' property

But setting both properties at the same command worked:

$ nmcli c mod vlan24-enp2s0 connection.slave-type bridge connection.master vibr24
$ nmcli con up br24
Connection successfully activated (master waiting for slaves) (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/50)
$ nmcli con show
NAME                    UUID                                  TYPE            DEVICE    
br24                    502643a8-1fbd-4881-9251-7bf1599bf05f  bridge          vibr24    
vlan24-enp2s0           70cb4491-0cfb-4f5c-a0b2-9cee71eb115a  vlan            enp2s0.24 
Connection 1  1e65cdef-c819-3c30-8fc8-23f4d13f7fe1  802-3-ethernet  enp2s0   

Checking this with brctl to confirm the VLAN is up & bound to bridge:

$ brctl show
bridge name     bridge id               STP enabled     interfaces
vibr24          8000.cafedefaced0       yes             enp2s0.24

So adapting this for your configuration, this one should work for you:

nmcli con add con-name vlan24-team0 type vlan id 24 dev team0
nmcli con add con-name br24 type bridge ifname vibr24 stp yes
nmcli con mod vlan24-team0 connection.slave-type bridge connection.master vibr24

PS: Just noticed this question are actually from 2015 and was bumped due to bounty by other user. Still my answer solves this issue, as I did reproduced this situation and had the same "No suitable device found for this connection." issue as OP before I've tried this solution.