3

I have a Linux box with one real (as opposes to virtual, a.k.a. aliased) Ethernet interface I can use (eth0 is used for other purposes - can't use it, nor can I add more NICs). Say it's eth1

I need to control some objects/entities via SNMP, so I set up a virtual Ethernet interface for each object, with its appropriate MAC address. I do this by (example for vif1):

ip -family inet link add link eth1 name vif1 address <the MAC addr> type macvlan
ip link set vif1 up multicast on
ip route del default dev vif1 table main /* enable the pings/TFTP going out! */
ip route add default via 192.168.1.1 table main proto static metric /* restore orig */

eth1,vif1,vif2,... all get an IP addresses from a single (remote) DHCP server. All these IP addresses are, of course, on the same IP subnet, say 10.11.1.0/24

Problem: ping from Linux box to the DHCP server (say 10.11.1.1) machine works. ping from the DHCP server machine to eth1 IP or any vif#X IP works, BUT (the issue, I suppose...) only eth1 responds to the ICMP packets (verified by ifconfig counters and by wireshark sniffing) This problem causes the inability to connect to the SNMP agents associated with vif interfaces' IP addresses.

I'm guessing that I need to set up internal routing so that IP packets reach their destination vif#X. I've tried adding an ip rule, with a new ip routing table, but probably didn't set it (the new table) correctly... Any one can tell me how (and preferably also why) to do this?

The Linux box runs Ubuntu9.04 and the DHCP-server runs Windows XP SP3

Improve this question
2
  • Why do you need a separate interface for each SNMP device?
    – Andrew
    Nov 18, 2011 at 5:01
  • Each SNMP agent should be identified, by the MIB-browser, as having a distinct different IP address. This is mandatory by the spec I'm following.
    – DavidG
    Nov 18, 2011 at 7:18

2 Answers 2

Reset to default
2

Solved it, finally: It's an ARP related issue.

  1. The DHCP server assigns an IP address to the virtual interface MAC address and sets this pair in the server's local ARP table
  2. The Linux box ties the new IP address to the virtual interface that requested it.
  3. PINGs are working on both ways:
  4. When pinging from Linux to server, it goes out through the real interface (which is on the same IP subnet)
  5. When pinging from server to Linux, again the real interface responds so it seems as if everything is OK...

BUT

When the server sends IP packets (in my case, SNMP messages) it uses the virtual interface's MAC address. When it reaches to the Linux box, the kernel just discards these frames, since it doesn't know how to forward them; Running Wireshark displays these messages since usually the interface is put in promiscuous mode

In order to make the SNMP messages reach the SNMP agent which is bind to the virtual interface, the IP packet must have the real interface's MAC address (I think that only then the kernel does VLAN routing, based on the IP address...)

The way to achieve this, is to send a gratuitous ARP request from the Linux' box real interface to the server, stating that the newly assigned IP address (to one of the virtual interfaces...) is "owned" by the real interface's MAC address. This updates the server's ARP table correctly.

BTW, this also explains why waiting some time before starting the SNMP traffic works: the server's ARP table entry is aged out, so the server sends an ARP-request which is responded correctly by the real interface

Improve this answer
3
  • I wonder why you actually used macvlan in the first place. Apparently, it has some strange side-effects so your traffic is not routed correctly internally. What you see by manipulating the ARP table is actually not a "magic fix" but the effect of something called "weak end host" on your host's side.
    – the-wabbit
    Feb 1, 2012 at 20:37
  • Thank you syneticon-dj! Didn't think that my fix is a "magical" one, just a fix that works. I'm using the macvlan option as the simplest (I'm aware of..) way to create a virtual interface that I can set its MAC address and get an IP address for it. Do you have another, better way (can be more complex, if it's better...)?
    – DavidG
    Feb 5, 2012 at 8:41
  • I don't know much about macvlan's behavior and I can't quite see why it is not working I understand it should reading the sparse documentation. But the way it is working for you now is depending on an operation mode's specific behavior only a few people are aware of. I would suggest using a different type of virtual interface (veth) along with bridging (brctl has been mentioned already) if you can't get macvlan alone to work "properly".
    – the-wabbit
    Feb 5, 2012 at 22:46
0

Why don't you set up a bridge-device? brctl addbr bridge Add the IP and MAC of the physical device to that bridge, move the device-without IP-to the bridge, then attach your VIFs to the bridge as well.

Improve this answer
3
  • Thanks Nils! I wasn't aware of such an option. Doesn't adding a macvlan type of interface (as I do) the same as adding a layer2 device, like a bridge-device? BTW, the IP address for the virtual NIC is obtained through DHCP, by invoking a DHCP client process on the interface - Will this work with a bridge-device as well?
    – DavidG
    Dec 28, 2011 at 14:50
  • The virtual interface I create is already "coupled" with the real Ethernet interface, so I guess they are already "bridged", no? Would you care, please, to explain the difference?
    – DavidG
    Dec 28, 2011 at 14:57
  • I don`t know about the macvlan option (I just tried to find it in my CentOS 5 man-page and came up blank). Your solution just sounded complicated and I am using bridges for the same purpose ever since I do virtualisation with XEN. I just googled a bit and found some complaints about macvlan not doing what it is supposed to do. So my guess is that it is still kind of experimental. A Bridge-Device behaves like a Switch - it even has a "learning" mode and you can enable STP if you like to.
    – Nils
    Dec 28, 2011 at 21:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .