On a decidated root server with 1+X public IP addresses in a hosted environment, running Debian Lenny with Xen 3.2, I want to install multiple domUs. Bridged network route is not an option due to hosting company requirements. They recommend routed setup but as far as I can understand this requires spending two public IPs on the dom0 which I can't affort.

In my setup, X domUs will have a public facing IP address and should be reachable from the network. Other domUs should be in private subnets (e.g. 10.0.. / 192.168..) and not reachable from the outside. domUs in the same private subnets should be able to reach each other but not domUs in other private subnets. A plus would be if all traffic (incl. the domUs with public IP addresses) were routed through the dom0 which could act as firewall (iptables?).

Is there anybody who has a similar setup as I and is willing to share some configuration files and tips?

  • 1
    Have the hosting company given a reason why bringing up multiple MACs with the bridging approach would be a problem? – Dan Carley Jun 4 '09 at 8:16
  • No, but it's the third hosting company I've been using over the years and they all disallowed more than one MAC per server. Guess it has to do with routing and security. – Steven Jun 4 '09 at 10:14

You should setup two bridges on dom0. You can use the standard entries in /etc/network/interfaces for this. Let's assume your real card is eth0 (and there's a DHCP server behind it) and you have bridge-utils installed. The file could look like this:

auto br0
iface br0 inet dhcp
    bridge_ports eth0
    bridge_maxwait 0

auto br1
iface br1 inet static
    bridge_ports none
    bridge_maxwait 0

You configure /etc/xen/xend-config.sxp with network-brigde and vif-bridge. In each domU config file, you select whether you want it to have direct external access (via br0) or if it should get only access via br1. For this you can use vif lines like those:

vif = ['bridge=br0']
vif = ['bridge=br1']

Of course, you still have to setup the NAT/masquerading over br1 and the network configuration of the domU should match (i.e. those on br0 should used DHCP and those on br1 should have a static IP in my example above).

  • I must not use a bridge as this would make the VMs visible on the hoster's network and would lock my server down. – Steven Jun 4 '09 at 7:35
  • Raphaël proposes putting the VMs on an internal bridge not connected to the external network. – David Schmitt Jun 4 '09 at 9:01
  • Yes, t least for those connected to br1. Those connected to br0 would indeed be visible on the hoster network but if they have public IP, it's the easiest solution. Otherwise, if you really must not expose their MAC, you have to look into Proxy ARP as pointed out in another comment. In that case, you probably have to take out eth0 from br0 and add routes for each public IP of the domU pointing to br0. And you have to enable proxy arp with "echo 1 >proc/sys/net/ipv4/conf/br0/proxy_arp" and "echo 1 >proc/sys/net/ipv4/conf/eth0/proxy_arp". Also enable ip_forwarding. Untested but feedback welcome. – Raphaël Hertzog Jun 5 '09 at 7:23

I have setup the exact same thing. What we did was NAT. This is definently what you are looking for. You need to have a script in the dom0 NATing the traffic of the public IPs to the appropriate private IPs. You can obviously apply firewall rules in the process.

For the NATing of the public IPs a single line is sufficient :

iptables -t nat -A PREROUTING -i eth0 -d PUBLICIP -j DNAT --to-destination INTERNALIP

Repeat this process for each public IP add firewall rules if you wish.

Masquerading the VMs :

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Bridged network route is not an option due to hosting company requirements.

I am just wondering, I have no idea if this will work, but have you investigated the possibility of using a proxy arp approach?

  • I have not, but it reads interesting. I will have to give it a try, I guess :) – Steven Jun 4 '09 at 7:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy