4

We have a number of XEN servers all running CentOS or RHEL. The VM's that they run are all required to be on their own VLAN for no other reason than the customer expects them to be. Long story short however, I can't change this right now.

We are also required to have bonding enabled on the interfaces. So to accommodat this we enslave eth1 and eth2 to bond0. We then create a seperate interface called bond0.VLANID where VLANID corresponds to the correct vlan; eg ifcfg-bond0.204

DEVICE=bond0.204
BOOTPROTO=static
ONBOOT=yes
VLAN=yes
BRIDGE=xenvlan204

Bridge to XEN

As you will see, we eventually have to bridge this out to XEN, and we do this by adding another interface called xenvlan204 (in this instance) which contains; ifcfg-xenvlan204

DEVICE=xenvlan204
BOOTPROTO=none
ONBOOT=yes
TYPE=bridge

XEN Vm Config

Finally in our XEN config for each VM, we add

vif = [ "bridge=xenvlan204" ]

This then allows the vm host to access that particular vlan

The Problem

We've noticed a few problems with this setup. One being that we currently create the interfaces manually. Which means if we add more vlan enabled interfaces and bridges we usually have to restart xend which is something I'm not so hot about. Also lower level staff have their heads melted by the number of interfaces and the risk of a mistake occurring is high.

Secondly, it can take sometime for a host to come up if it has a number of vlan taged interfaces.

Thirdly, its just not scaling well on the management aspects

The Question

Is there a better more flexible way to do this (in particular with Xen that ships with centos 5.3, 5.4 and 5.5 as we have to support all three) that leverages either scripting or other solutions to allow an arbitrary amount of interfaces to be created when a vm is instanced.

Your advise and expertise is more that welcomed.

1 Answer 1

1
+50

Not an expert on Xen, but what you're doing with the bridged interfaces, plus the vlan tag, is quote OK, as long as you stay away from bonding mode 0 and 6 (those are known to have issues with bridges on top of them).

One more thing, since you're using the RHEL based Xen, you must be also using libvirt. As far as I have seen it using KVM, there was never any need to reload anything to see a new bridge available

Lastly, as I mentioned, I'm no Xen expert, so I might be shooting at nothing here, but can't xend be reloaded instead of restarted?

4
  • Thanks, however alas I can't move to KVM as of yet (Not without going through massive change controls/tests etc). I was considering altering the vif-bridge scripts to create the interfaces etc as needed however I still need to restart/reload xend, which even if its a reload is a bit crappy to have to do. The other big issue is quite simply the number of interfaces. I could end up in a situation where I have dozens of them, 1 for bond one for bridge. I see the junior admins getting lost there very quickly and worry about a mistake occurring.
    – AJ01
    Jul 22, 2010 at 10:11
  • if you're looking for an easy to use solution, you'll really need to abandon non-graphical tools :) take a look at RHEV for well managed KVM hosts, and at Citrix's stuff for Xen.
    – dyasny
    Jul 22, 2010 at 14:14
  • Hear you about the GUI, however what I still need to do till a) finance and b) testing can be completed on new infra is try and come up with a some easier way to do above.
    – AJ01
    Jul 22, 2010 at 23:03
  • well, scripting out standard sets of ifcfg files shouldn't be hard. oh, and btw, don't forget you can do the vlan tagging part inside the VMs or on the switch, so that saves a bit of extra work
    – dyasny
    Jul 23, 2010 at 5:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .