I'm looking at implementing a systems using a number of tagged and a native vlan connected to a server over a a/p bonded interface. The untagged vlan is for physical machine access, the tagged vlans are connected to bridges and then to QEMU VM's inside the machine. Hopefully this plan is fine, but I'm trying to implement a crippled version of this in a dev environment due to a lack of underlying network config in this location where I just have the same single vlan delivered to the machine on a tag AND plain. I'm nto clear if this is going to work (and that I should just be confident that it will work using different vlans) as I'm seeing odd things like a vm is arping out over the vlan out to the core switch, but the arp reply is coming back on the untagged interface. Now an ARP reply is unicast right? So it's a deliberate thing to send the ARP response on the untagged interface, and not a case that a broadcast response isn't being passed on the tagged side... i.e. there's some underlying logic pushing it that way. Something about the MACs somehow? This is on a CentOS 5.5 machine, vlan's from vconfig. (I've seen reference to the Linux mac-vlan project work, but that's not available here by default.)

so 1) Should having the SAME vlan tagged and untagged work?

2) Will different tagged vlans to the untagged interface work nice and easily?

  • can you maybe add a set of your current ifcfg scripts and a sketch of the networking diagram, so it's clearer for people to understand your question? – dyasny Jan 12 '11 at 10:05

1) This seems like an extremely broken configuration so I'm not surprised you're getting strange behavior with the ARP replies - I don't think it would be unexpected behavior for replies sent from the VLAN interface to arrive back at the untagged interface - they are on the same physical network with the same MAC address after all.

More importantly..

2) This configuration is quite common and will work well. In the last few days I've brought up a couple of KVM hosts running RHEL6 with pretty much the exact config you describe - untagged VLAN to the NIC for management and tagged interfaces for all the bridged virtual interfaces.

  • When I designed this I was happy it was fairly mundane and built on very boring stable bulding blocks, just a number of stages, but the "establishment" IT staff where I'm working think I'm just insane. The feeling is mostly mutual though! – Chris Phillips Jan 12 '11 at 10:34
  • I don't deny that my test setup is horrible, but I have to work with what's available. As I think I see it, CatOS on the switch side doesn't have an interface per vlan unlike IOS, so it knows that a vlan in on a port, and the CAM tables don't identify the vlans as different interfaces. So there's no way to use the tagged version of the vlan. I mean, the MAC of the VM i'm testing is 1) on vlan X, port y, but no further information about whether it's on the tagged or untagged path. Hence I don't see the ARP replies on the tag. But with different vlans they should be fine. – Chris Phillips Jan 12 '11 at 10:43
  • The alternative would be installing an additional NIC in the host machine for each network you want to connect it to, good if your applications hit the network really hard but otherwise a bit of a waste of network cards (and switch ports). I'm supprised the IT people where you are haven't seen this configuration before - it's pretty common with virtualised setups, for a reference VMWare ESX allows you to create (and supports) this sort of configuration via a nice GUI. – James Yale Jan 12 '11 at 10:44
  • It's the port density that's not there. This is scoped for up to 200 boxes, and they (apparently) need to be directly patched into Catalyst 6513's, so port density is REALLY expensive... We have 4 subnets in my design, some people reckon that all traffic (management and internet and database...) should just be dumped onto a single vlan, nice and insecure and all. – Chris Phillips Jan 12 '11 at 10:51

let me understand this, besides my comment to the top post, you're tagging an interface on the switch side, so the traffic is already VLAN'd, and then you're adding a tag in the OS, with the same tag number. If I understood the question correctly, this means you're nesting VLANs, not just tagging

If you have a tagged network attached to a bridged interface, and the VMs are plugged intot hat bridge, all their traffic will be also tagged, whether the switch does it or the tag under the bridge... but I'll wait for an update, to better understand the question

  • No nested tags, just using tags on both sides of the link, but the main point is that it's the same vlan being tagged as on the native vlan. It is the vlan interfaces (i.e. bond0.10 not just bond0) that go onto the bridge, so each bridge only holds one vlan, which, by the time it reaches the bridge, is untagged again. – Chris Phillips Jan 12 '11 at 10:40
  • if the bridge is attached to a tagged bond/eth, all traffic it relays from the plugged in VMs will also be tagged – dyasny Jan 12 '11 at 11:04
  • yep, that's the point. – Chris Phillips Jan 12 '11 at 13:33

I'd avoid this mixed configuration, native vlan setting are a crutch to help out in temporary situations or where you have no vlan-aware stack. I'd be more than tempted to just make all your traffic tagged, at least you know where you are then and you shouldn't see any cam table anomolies.

  • Well a major concern here is that the physical boxes are rebuilt regularly using DHCP / PXE so they will need a native network to do this over. – Chris Phillips Jan 12 '11 at 10:33
  • Understood, that's about the one good reason to do this with modern OS's - maybe just a native for the PXE vlan then? – Chopper3 Jan 12 '11 at 18:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.