1

While I am fairly new to XEN (running under Ubuntu 14.04) I've not had any real issues until now. I have set my networking using bridging and that has worked well for many different machines until now. Normally I’m able to find my answers somewhere, but this time I've come up short and looking for some help.

I have client VM(DomU) that I’m trying to setup with a large number of IPs and it is giving me lots of grief. Here is the setup:

Host machine has two NICs, one is used for private (10.10.10.x) xenbr0 and the other is just bridged for public IPS (say x.y.z.*) xenbr1:

Below is my /etc/networking/interfaces file from the host:

iface eth0 inet manual
auto xenbr0
iface xenbr0 inet static
    bridge_ports eth0
    address 10.10.10.112
    netmask 255.255.255.0
    network 10.10.10.0
    broadcast 10.10.10.255
    gateway 10.10.10.1

auto eth1
iface eth1 inet manual
auto xenbr1
iface xenbr1 inet manual
       bridge_ports eth1
       bridge_maxwait 1

In the client VM also uses two bridged nic /etc/networking/interfaces:

auto eth0
iface eth0 inet  static
    address x.y.z.170
    netmask 255.255.255.240
    broadcast x.y.z.175
    gateway x.y.z.161

auto eth1
iface eth1 inet static
    address 10.10.10.155
    netmask 255.255.255.0
    broadcast 10.10.10.255
auto eth0:1                    
iface eth0:1 inet static       
        address x.y.z.162      
        netmask 255.255.255.240
        broadcast x.y.z.175    
auto eth0:2                    
iface eth0:2 inet static       
        address x.y.z.163      
        netmask 255.255.255.240
        broadcast x.y.z.175    
auto eth0:3                    
iface eth0:3 inet static       
        address x.y.z.164      
        netmask 255.255.255.240
        broadcast x.y.z.175    
etc…. up to 8 alaises.    

In my xen config file I have: vif = [ 'bridge=xenbr0', 'bridge=xenbr1' ]

I plan to remove the local IPs from the VM once everything is setup and working.

I’m using aliases since I’m trying to move this from a bare metal machine to a VM one. Eventually the overuse of IPS will be deal with, but for various reasons I can’t do that yet. Anyways, when I boot the VM only a few of the IPs are pingable unless I do an arping -U -I eth1 x.y.z.162, etc for each IP address. Then things seem to at least respond, but the routing doesn't really work correctly. I can ssh and browse to the IPs via http, but virtual sites aren't working correctly (ie it only sends me to one and ignores the others) and restarting bind and some other services don't respond correctly. It is as if the routing is not setup correctly and while the IPs respond, it isn't truly the IP requested that is replying, but another IP.
This is however only on SOME of the extra IPS. When I reboot, the routing that the arp command fixed, is lost. Then Bind and a few other services on the VM won’t work. I do the arp command and things work again.
I think the arping is just a band-aid, but not fixing the real issue. Since the exact same system works on a bare metal machine I'm almost positive the issue is with my XEN network setup and not the switch setup or OS I'm trying to migrate.

To further isolate the issue I setup a clean Ubuntu 14.04 guest with multiple IPs using the more modern (and not depreciated IP aliasing) setup show below in the VMs /etc/networking/interfaces file:

 auto eth0
allow-hotplug eth0
iface eth0 inet static
      address x.y.z.110
      netmask 255.255.255.240
      broadcast x.y.z.175
      gateway x.y.z.161
iface eth0 inet static
      address x.y.z 162
      netmask 255.255.255.240
      broadcast x.y.z.175
iface eth0 inet static
      address x.y.z 163
      netmask 255.255.255.240
      broadcast x.y.z.175
   and on with the other IPS

All with the same results. What am I missing? Any assistance would be appreciated.

1

I figured out what my problem was and I'm posting it here should anyone runs into a similar issue.

In my Xen config I did not put in a MAC address. IE I had:

vif = [ 'bridge=xenbr0', 'bridge=xenbr1' ]

When I needed something like:

vif = [ 'bridge=xenbr0', 'mac=00:11:5e:5n:2c:1c,bridge=xenbr1' ]

The routing for the MAC address didn't expire right away so running this ARP command for each IP fixed the routing: arping -U -I eth1 x.y.z.162

I'm guessing each time the VM booted up it had a new MAC address and only some of the routing tables were updated when the VM booted.

Now that the MAC address isn't changing all is good.

0

Try swapping interface configurations in DomU. According to your Dom0 configuration, you should use eth0 for private and eth1 for public IP's in your DomU.

  • I've tried swapping eth0 and eth1 without any changes. – Derf Jul 5 '15 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.