0

I'm trying to get two virtual machines (running CentOS) to talk to host (Ubuntu 10.04) using VDE and the outside network.

So far I'm running into an ARP issue.

On Host I have a physical eth0 and a virtual tap0 interfaces bridged by br0. All on the same subnet.

br0       Link encap:Ethernet  HWaddr 78:e3:b5:90:88:df  
          inet addr:172.16.1.3  Bcast:172.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:1381 (1.3 KB)

br0:1     Link encap:Ethernet  HWaddr 78:e3:b5:90:88:df  
          inet addr:172.16.1.4  Bcast:172.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0      Link encap:Ethernet  HWaddr 78:e3:b5:90:88:df  
          inet addr:172.16.1.1  Bcast:172.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2362 (2.3 KB)
          Interrupt:33 Base address:0x6000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:778 errors:0 dropped:0 overruns:0 frame:0
          TX packets:778 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:62832 (62.8 KB)  TX bytes:62832 (62.8 KB)

tap0      Link encap:Ethernet  HWaddr b6:22:43:93:ed:60  
          inet addr:172.16.1.2  Bcast:172.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15 errors:0 dropped:3 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:0 (0.0 B)  TX bytes:1980 (1.9 KB)

When I ping this Host Ubuntu machine from another machine (or from the Guest), the arp cache entry is stored as a br0 entry, so the Ubuntu machine never responds to ping.

Address                  HWtype  HWaddress           Flags Mask            Iface
172.22.64.4                      (incomplete)                              eth0
172.22.64.4              ether   00:25:b3:0f:0b:14   C                     br0

Here's the interfaces file that actually works for setting up the IP addresses on all interfaces: auto lo iface lo inet loopback

auto eth0
iface eth0 inet manual
        address 172.16.1.1
        netmask 255.0.0.0
        gateway 172.16.1.255
        post-up ifconfig eth0 172.16.1.1 netmask 255.0.0.0

auto tap0
iface tap0 inet static
        address 172.16.1.2
        netmask 255.0.0.0
        gateway 172.16.1.255
        pre-up vde_tunctl -t $IFACE
        post-up ifconfig tap0 172.16.1.2 netmask 255.0.0.0
        post-up vde_switch -t $IFACE -s /tmp/vde-$IFACE -d -g rhuser -m 664
        post-down vde_tunctl -d $IFACE

auto br0
iface br0 inet static
        address 172.16.1.3
        netmask 255.0.0.0
        gateway 172.16.1.255
        post-up ifconfig br0 172.16.1.3 netmask 255.0.0.0
        post-up ifconfig br0:1 172.16.1.4 netmask 255.0.0.0
        bridge_ports eth0 tap0

I've tried turning off ARP on br0 by ip link set br0 arp off. I've also tried setting things like arp_ignore for br0. In both cases Ubuntu never responds to ARP requests at all. And this is my first question - why does eth0 not respond to ARP requests when a bridge exists. It does well when bridge is commented out from interfaces.

  1. Is arp_filter useful in this situation? It's documentation says "you must use source based routing for this to work". Can someone throw me a link that explains how I set up "source based routing" (I'm pretty new to the topic).

  2. Can arptables or ebtables be used to control this behavior? Looking at their documentation, I see info on how to react to apr requests and when to respond to them, but not much regarding apr cache behavior.

  3. Generally, should I be doing something with sysctl and ARP or should I set my IP assignments differently? I see guides on the net that tell you how to do this trouble-free, but they all assign guests to a separate subnet, making bridge as a gateway. That would not work for me - I need all devices to be on the same subnet.

Any idea would do as long as:

  • VMs are on the same subnet as outside network.
  • VMs can talk to host in order to have NFS, etc.
  • Host does not necessarily need to talk to outside network.

  • 1

    1 Answer 1

    2

    This is the so called "arp problem."

    http://www.ssi.bg/~ja/#arp_announce

    See your linux/Documentation/networking/ip-sysctl.txt file for more info about the arp_announce and arp_ignore device flags.

    I think you want arp_announce=1 for all interfaces.

    See also (generally):

    http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.arp_problem.html

    2
    • So far tried the following: arp_ignore=1 arp_announce=2 on either eth0 or on br0 or on both - no change; arp_ignore=1 arp_announce=2 on all - now Ubuntu does not respond to arp at all; arp_announce=1 on all as per your suggestion - no change Jan 13, 2012 at 17:43
    • noarp module - couldn't compile, I guess it's ancient... Jan 13, 2012 at 17:53

    You must log in to answer this question.

    Not the answer you're looking for? Browse other questions tagged .