We have this situation: A router, which is an Ubuntu VM, has multiple VLAN interfaces on the same physical NIC. Those VLAN interfaces are dynamically configured and torn down and attached to networks where machines can be added or removed by users.
Sometimes we have a situation where, for example, on VLAN 1 the router has the IP 10.0.0.254 and a neighbor machine has the IP 10.0.0.1. While, at the same time, on VLAN 2 the router has the IP 10.0.0.1 and a neighbor machine has the IP 10.0.0.2.
When, on VLAN 1 10.0.0.1 is sending an ARP asking "who has 10.0.0.254 tell 10.0.0.1" the router does not respond. Sniffing traffic on the router shows that the ARP requests arrive with proper VLAN tagging but are discarded.
Does the router discard the ARP message thinking that it arrived in the wrong interface?
A few notes:
- On VLAN 2 there are no ARP problems.
- Replacing the IP of the neighbor VM on VLAN 1 (10.0.0.1) to 10.0.0.3 solves the problem.
- As I mentioned, both IP's (10.0.0.254 and 10.0.0.1) are on the same MAC address.
- arp_filter is set to 0 on all interfaces
- the router's routing table does not refer all those dynamic interfaces. we use iptables and fwmarks along with IP rules and custom routing tables (all are dynamically set by software) to route traffic separately according to the VLAN it is originated.