Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I've done a lot of reading about VLAN tagging in KVM bridged networks, but am getting very confused as to what needs to be tagged (the physical network devices, or the bridges attached to those devices) and whether or not the VM guest needs to tag it's internal device or this happens on the host side when the guests virtual NIC is attached to a tagged bridge.

Disclaimer: I make some assumptions below based on bits and pieces I'd picked up from a few blog entries/tech pages. Like a lot of things, there seems to be many ways of accomplishing the one task. If I'm just completely incorrect and someone knows of a good guide, send me the URL and I'll revise what I've done and come back if it doesn't work

I have a number of KVM virtual hosts all (including their guests) currently connected to the one network.

I'd like to segment this so that there's a logically separate network in use by a few guests (but not on all obviously). I've done a lot of reading into how to accomplish this, but have some basic questions about the actual process.

The setup:

kvmhost1

  • eth0
  • br0

    kh1guest1

    • eth0 (connected to kvmhost1 br0)

    kh1guest2

    • eth0 (connected to kvmhost1 br0)

kvmhost2

  • eth0
  • br0
  • br0.10 (tagged VLAN10)

    kh2guest1

    • eth0 (connected to kvmhost2 br0)

    kh2guest2

    • eth0 (connected to kvmhost2 br0.10)

kvmhost3

  • eth0
  • br0
  • br0.10 (tagged VLAN10)

    kh3guest1

    • eth0 (connected to kvmhost3 br0)

    kh3guest2

    • eth0 (connected to kvmhost3 br0.10)

I'm making some assumptions here:

  • for the whole thing to work, all hosts/guests need 802.1q enabled
  • all hosts with guests that need access to the tagged VLAN need an additional bridge, tagged onto the VLAN
  • any guest VM that needs access the tagged VLAN needs it's network connected to the tagged network bridge (br0.10 on the host(s))

Excuse my ignorance here, but my knowledge of VLANs are quite limited, and I've really only dealt with them on a physical level thus far(NIC into VLAN'ed switch port)

Any help would be great, or even a guide as to the proper way of accomplishing this.

share|improve this question
    
Don't you mean br0.10? The colon would cause something completely different to happen, which is probably not what you want. –  Michael Hampton Jun 5 '13 at 17:18
    
ah yes, sorry oversight on my side meant .10! –  kwiksand Jun 5 '13 at 22:05
add comment

3 Answers

The best option is to do as follows:

  • tag the interface (or bond) - e.g. eth0 -> eth0.100 (or eth0+eth1 -> bond0 -> bond0.100)
  • build bridge on top of the tagged interface
  • plug VMs into the bridge

This way all traffic to and from the VMs will be tagged. For untagged traffic it is best to create another bridge on the untagged interface and plug another VM NIC into that

share|improve this answer
    
eth0+eth1 -> bond0 -> bond0.100 is ok, but vlan-tagged virtual interfaces like eth0.100 shouldn't be bonded –  suprjami Jun 9 '13 at 3:47
    
bonding is optional, if you do bond, you should tag the bond, and not bond tagged interfaces –  dyasny Jun 9 '13 at 4:59
    
@dyasny bridging guest network with eth0 (untagged) always gave me the feeling that it gets access to all the VLANs. I never took my time to test it and just never used native vlan for VMs. Is the guest limited to untagged traffic with that setup? –  skara Jun 9 '13 at 7:09
    
That's correct, but do you really want the guest to see ALL the traffic? I know I wouldn't. This is why it is generally not done, and there are systems that wouldn't let you bridge an untagged interface if it has tags on it as well –  dyasny Jun 9 '13 at 19:11
add comment

What you've setup is logically fine.

These guests will be able to see each other because they're in native vlan:
kh1guest1, kh1guest2, kh2guest1, kh3guest1

These guests will be able to see each other because they're in vlan 10:
kh2guest2, kh3guest2

Your switch admin may need to configure your switchports as "trunk ports" to be able to pass the vlan tag which you're adding. Let them know what you've setup and your requirements for having a vlan tag.

If you're in a large managed network then you might need to ask for a vlan number to be allocated to you, because something else already has vlan10.

Think of the vlan tag as a "wrapper" around an ethernet frame. You just need to define one place to add/remove the wrapper. All the following are valid, the bold interface shows where the tagging is done:

  • wire -> host eth0 -> host br0 -> host br0.10 -> guest eth0
  • wire -> host eth0 -> host eth0.10 -> host br10 -> guest eth0
  • wire -> host eth0 -> host br0 -> guest eth0 -> guest eth0.10

If you want to bond physical interfaces on the host, then you cannot add the vlan-tagged interfaces to a bond. So these would be valid:

  • wire -> host ethX -> host bond0 -> host bond0 -> host br0 -> host br0.10 -> guest eth0
  • wire -> host ethX -> host bond0 -> host bond0.10 -> host br10 -> guest eth0
  • wire -> host ethX -> host bond0 -> host bond0 -> host br0 -> guest eth0 -> guest eth0.10

And this is wrong:

  • wire -> host ethX -> host ethX.10 -> host bond -> and so on
share|improve this answer
    
While these will, in a way, work, they are mostly wrong, the reason being - it's going to become increasingly hard to manage, especially if you need to perform live migrations. The most optimal sert of settings is to have tagging done on the interface that is under the bridge. And btw, tagging inside the guest, while it does work, has proven to be problematic in the current virtio implementation –  dyasny Jun 9 '13 at 5:08
add comment

The simple rule is that a connection should either be untagged at both ends (host side and guest side) or tagged at both ends.

Guests usually do not need tagging support. For now, ignore the fact that you are in a virtualized environment. How would each "guest" be set up if it were a physical machine? Then implement the same in KVM.

What you probably want on kvmhost2 and kvmhost3 is two bridges, br0 and br1. br1 can have eth0.10 as a member. Then, kh2guest2 and kh3guest3 can connect their eth0 to their respective host's br1.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.