Xen 3.2 networking: private and public hosts?

Question

I've got Xen 3.2 running on Debian Lenny, and I'm essentially trying to recreate the setup I had on another server that was running OpenVZ.

My dom0 has a static IP address and is Internet facing. Let's say for example, I have 4 domUs. I want dom1 and dom2 to be public facing on the Internet with public IP addresses. I want dom3 and dom4 to have private network addresses (192.168.x.x) yet able to be reached from dom0, dom1, dom2.

This was a snap with OpenVZ, but unfortunately I can't find any good examples of combination setups like these. Either all the domUs are public or they're all private. Any suggestions?

Answer 1

You could simply create a virtual bridge, which is not connected to any physical interfaces for your domU3 and domU4. You other 2 domU's can have an interface on each bridge (physical and virtual) and you can use NAT on dom0 to allow domU3/4 to access the internet through the virtual bridge, by assigning it an IP in dom0. RedHat's libvirt does just this by creating a default "virbr0" which can be used for creating a private domU LAN.

With debian you can easily setup a bridge on startup by: a. installing "bridge-utils" (which you most likely already have if you have xen installed) b. adding something along the following to /etc/network/interfaces:

auto virbr0
iface br0 inet static
address 192.168.0.10
netmask 255.255.255.0
gateway 192.168.0.1

and adding an interface to your domU's which is connected to virbr0 in your domU configuration

vif = [ "bridge=virbr0" ]

Answer 2

If you have 2 NICs on the box you can use on for public and one for private. My typical Xen configuration has the dom0 only having a private IP but has the public NIC setup to bridge. When you build the domU you only give it the bridge for the public or private network.

You need to have 2 NICs to do this, I'm not sure you could do it with 1 NIC because of the way Xen bridges the NIC. You will have to change the network-script setting in the xend-config.sxp to run a routine that will setup both bridges. I create a script called xen-bridge and have it call the default network-bridge script multiple times for each NIC that needs to be bridged.

Answer 3

There's several ways you could accomplish this. The way I have done it on my network is to use VLAN tagging on the switch ports, and have a large number of bridges and VLAN interfaces on each Xen host. I then assign the various VM's to a different bridge depending on which VLAN I want them to be hosted on. The router then takes care of the VM's being reachable from each other.

The other way you could handle it is through using iptables on the Dom0. All of the traffic from the DomU passes through the iptables stack of the Dom0, so you could write all sorts of rules there to send the packets around as necessary.