Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a Linux box with one real (as opposes to virtual, a.k.a. aliased) Ethernet interface I can use (eth0 is used for other purposes - can't use it, nor can I add more NICs). Say it's eth1

I need to control some objects/entities via SNMP, so I set up a virtual Ethernet interface for each object, with its appropriate MAC address. I do this by (example for vif1):

ip -family inet link add link eth1 name vif1 address <the MAC addr> type macvlan
ip link set vif1 up multicast on
ip route del default dev vif1 table main /* enable the pings/TFTP going out! */
ip route add default via 192.168.1.1 table main proto static metric /* restore orig */

eth1,vif1,vif2,... all get an IP addresses from a single (remote) DHCP server. All these IP addresses are, of course, on the same IP subnet, say 10.11.1.0/24

Problem: ping from Linux box to the DHCP server (say 10.11.1.1) machine works. ping from the DHCP server machine to eth1 IP or any vif#X IP works, BUT (the issue, I suppose...) only eth1 responds to the ICMP packets (verified by ifconfig counters and by wireshark sniffing) This problem causes the inability to connect to the SNMP agents associated with vif interfaces' IP addresses.

I'm guessing that I need to set up internal routing so that IP packets reach their destination vif#X. I've tried adding an ip rule, with a new ip routing table, but probably didn't set it (the new table) correctly... Any one can tell me how (and preferably also why) to do this?

The Linux box runs Ubuntu9.04 and the DHCP-server runs Windows XP SP3

share|improve this question
    
Why do you need a separate interface for each SNMP device? –  Andrew Nov 18 '11 at 5:01
    
Each SNMP agent should be identified, by the MIB-browser, as having a distinct different IP address. This is mandatory by the spec I'm following. –  DaveG Nov 18 '11 at 7:18
add comment

2 Answers

Solved it, finally: It's an ARP related issue.

  1. The DHCP server assigns an IP address to the virtual interface MAC address and sets this pair in the server's local ARP table
  2. The Linux box ties the new IP address to the virtual interface that requested it.
  3. PINGs are working on both ways:
  4. When pinging from Linux to server, it goes out through the real interface (which is on the same IP subnet)
  5. When pinging from server to Linux, again the real interface responds so it seems as if everything is OK...

BUT

When the server sends IP packets (in my case, SNMP messages) it uses the virtual interface's MAC address. When it reaches to the Linux box, the kernel just discards these frames, since it doesn't know how to forward them; Running Wireshark displays these messages since usually the interface is put in promiscuous mode

In order to make the SNMP messages reach the SNMP agent which is bind to the virtual interface, the IP packet must have the real interface's MAC address (I think that only then the kernel does VLAN routing, based on the IP address...)

The way to achieve this, is to send a gratuitous ARP request from the Linux' box real interface to the server, stating that the newly assigned IP address (to one of the virtual interfaces...) is "owned" by the real interface's MAC address. This updates the server's ARP table correctly.

BTW, this also explains why waiting some time before starting the SNMP traffic works: the server's ARP table entry is aged out, so the server sends an ARP-request which is responded correctly by the real interface

share|improve this answer
    
I wonder why you actually used macvlan in the first place. Apparently, it has some strange side-effects so your traffic is not routed correctly internally. What you see by manipulating the ARP table is actually not a "magic fix" but the effect of something called "weak end host" on your host's side. –  the-wabbit Feb 1 '12 at 20:37
    
Thank you syneticon-dj! Didn't think that my fix is a "magical" one, just a fix that works. I'm using the macvlan option as the simplest (I'm aware of..) way to create a virtual interface that I can set its MAC address and get an IP address for it. Do you have another, better way (can be more complex, if it's better...)? –  DaveG Feb 5 '12 at 8:41
    
I don't know much about macvlan's behavior and I can't quite see why it is not working I understand it should reading the sparse documentation. But the way it is working for you now is depending on an operation mode's specific behavior only a few people are aware of. I would suggest using a different type of virtual interface (veth) along with bridging (brctl has been mentioned already) if you can't get macvlan alone to work "properly". –  the-wabbit Feb 5 '12 at 22:46
add comment

Why don't you set up a bridge-device? brctl addbr bridge Add the IP and MAC of the physical device to that bridge, move the device-without IP-to the bridge, then attach your VIFs to the bridge as well.

share|improve this answer
    
Thanks Nils! I wasn't aware of such an option. Doesn't adding a macvlan type of interface (as I do) the same as adding a layer2 device, like a bridge-device? BTW, the IP address for the virtual NIC is obtained through DHCP, by invoking a DHCP client process on the interface - Will this work with a bridge-device as well? –  DaveG Dec 28 '11 at 14:50
    
The virtual interface I create is already "coupled" with the real Ethernet interface, so I guess they are already "bridged", no? Would you care, please, to explain the difference? –  DaveG Dec 28 '11 at 14:57
    
I don`t know about the macvlan option (I just tried to find it in my CentOS 5 man-page and came up blank). Your solution just sounded complicated and I am using bridges for the same purpose ever since I do virtualisation with XEN. I just googled a bit and found some complaints about macvlan not doing what it is supposed to do. So my guess is that it is still kind of experimental. A Bridge-Device behaves like a Switch - it even has a "learning" mode and you can enable STP if you like to. –  Nils Dec 28 '11 at 21:35
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.