eth1 doesn't come up at boot on dom0

Question

I am trying to install Xen 3.2 on Debian Lenny on a server with 2 NICs, one connected to the private LAN and the other to the Internet. This server has been in operation in a non-Xen state for a while and everything works fine. With Xen, however, networking isn't coming up properly. I'm having a hard time figuring out the proper way to configure this, as there is a lot of conflicting advice on the Xen site/wiki, mailing lists and even here.

Here are the basics: eth0 is the private network, eth1 is the public Internet. Ultimately, I want dom0 (and all domU's) to have access to both interfaces. There is currently an iptables script in place (from pre-Xen use) that sets up a firewall and port forwarding for other client machines on the private network. So even before I worry about installing a domU, I'm trying to get things working for bridging, knowing that's where I need to be.

The Xen Networking wiki and the Debian Xen wiki both point me toward using Xen's bridging scripts, but when I do this, eth1 does not come up automatically at boot time, so dom0 does not have access to the public network. It comes up fine when I sudo ifup eth1 manually. Note that I'm not even worrying about domU's yet, and I'm well aware that the out-of-the-box Xen bridging won't do anything for eth1 <-> domU.

This ServerFault question advises against using the Xen bridging scripts. I tried this configuration and dom0 can't access either private or public networks.

This mailing list thread says that the Debian bridge-utils package should do everything I need, but it seems to imply that if I want my dom0 to be on the public network, I have to assign an address to both the real eth1 as well as the bridge. So I need to burn two public addresses in this case?

/etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth1
allow-hotplug eth1
iface eth1 inet static
    address 24.123.138.34
    netmask 255.255.255.248
    network 24.123.138.32
    broadcast 24.123.138.39
    gateway 24.123.138.33
    # dns-* options are implemented by the resolvconf package, if installed
    dns-nameservers 127.0.0.1
    dns-search lightyearsoftware.com

auto eth1:0
iface eth1:0 inet static
    address 24.123.138.36
    netmask 255.255.255.248
    network 24.123.138.32
    broadcast 24.123.138.39
    gateway 24.123.138.33

# Internal network
auto eth0
allow-hotplug eth0
iface eth0 inet static
    address 192.168.1.1
    netmask 255.255.255.0
    network 192.168.1.0
    broadcast 192.168.1.255
#   mtu 9000    # memory alloc failures with 2.6.26-2 e1000 driver

#auto br-eth0 br-eth1
#iface br-eth0 inet manual
#   bridge_ports eth0
#iface br-eth1 inet manual
#   bridge_ports eth1

/etc/xen/xend-config.sxp uses (network-script network-dummy) when the last 4 lines of the interfaces file are uncommented, (network-script network-bridge) otherwise.

I'm more than happy to provide any other diagnostic information that can help.

Answer 1

"The Xen Networking wiki and the Debian Xen wiki both point me toward using Xen's bridging scripts"

The problem with wikis is that any idiot can edit them -- and they usually do. The Debian Xen wiki page is wrong, so very, very wrong; I don't even have the heart to look at the Xen networking wiki. There used to be useful networky docs in the README.Debian for the xen-utils package, but they seem to have gone walkabout at some stage.

My answer in Bringing up network interface without IP configured in Debian, for XEN dom0 probably won't work for you because it's solving a very different problem. You should probably re-read the question to discover the differences.

That mailing list thread is full of gibberish (and Nabble's usual brain-meltingly stupid formatting) so I'm not going to try and work out exactly where you're seeing a need to put an IP address on both the interface and the bridge, but I can assure you that you don't need to do that -- just an IP address on the bridge will be perfectly sufficient.

The simple way to fix your network config is probably to:

1. Set (network-script network-dummy) and then chattr +i /etc/xen/xend-config.sxp (to discourage you from trying to fiddle with it any more)
2. Delete the br-* stanzas you have currently
3. Delete the gateway line from the eth0:1 stanza
4. /allow_hotplug/d
5. s/eth0/br0/; s/eth1/br1/
6. Add bridge_ports eth0 to your now-br0 config
7. Add bridge_ports eth1 to your now-br1 config
8. /etc/init.d/networking restart