Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

We've got AT&T U-Verse internet service, which has an extremely boneheaded DSL gateway.

We have 5 IPs (netmask 248), but the gateway is incapable of doing anything other than a single IP -> single MAC address mapping.

We have a single firewall machine, and we redirect different IP/port combos to different places inside a DMZ.

Our solution so far is to have a VMWare virtual machine on the firewall with 4 additional NICs in it, to get the other 4 IP addresses... however we have a problem.

The gateway is basically doing an ARP ping to see if the IP is responding on the expected MAC. With 4 NICs all on the same LAN, linux is responding to ARP requests for ALL IPs using a single interface. That's not what the gateway is expecting, and it's messing up the 3 other NICs. The gateway refuses to route incoming traffic for IPs where the ARP ping results aren't the expected MAC.

How can we get the ARP replies for eth0's IP to go out eth0, eth1's IP to go out eth1, etc?

EDIT

Christopher Cashell's response does not work in this situation. I had great hopes reading it, but... nope.

EDIT 2

Solved! See my answer below.

share|improve this question
    
PS - no 'drop uverse' comments... U-Verse is 18mbps, vs. everything else's 3mbps or a very unreliable 10mbps via cable –  darron Jun 8 '09 at 22:46
    
@dblack: You forgot to add your answer :) –  Mihai Limbăşan Jun 9 '09 at 4:56
    
The api.recaptcha.net site was down for a little while. The answer is up now. –  darron Jun 9 '09 at 5:36

5 Answers 5

Your chosen solution works, but there are alternatives that don't involve arptables. (Christopher Cashell was on the right track, originally, but he was off by a smidge.)

In short, you want to set these parameters:

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

These should be available when running a modern, 2.6 series Linux kernel. Check and make sure you that '/proc/sys/net/ipv4/conf/*/arp_announce' and /proc/sys/net/ipv4/conf/*/arp_ignore' exist on your system.

The 'arp_filter' parameter only works when your various IP addresses share a LAN segment but use different IP subnet. If they share the IP subnet, too, you need to use 'arp_ignore' and 'arp_announce', as above.

(I believe you may also need to set 'arp_filter' back to '0', too.)

share|improve this answer
    
What happened to arp_filter? It was an early (2003?) solution to the ARP problem, but it doesn't seem to work as described, at least on Centos 5. arp_ignore and arp_announce seem to be good. –  pcapademic May 22 '10 at 14:49
up vote 8 down vote accepted

Okay, here's the solution. First, a recap:

Here's my basic network plan:

 eth0 10.10.10.2 netmask 255.255.255.248
 eth1 10.10.10.3 netmask 255.255.255.248
 eth2 10.10.10.4 netmask 255.255.255.248
 eth3 10.10.10.5 netmask 255.255.255.248

All the interfaces overlap. This is technically wrong, and the source of all my woes... but I have to do it because of this dumb residential gateway.

First, broadcast ARP requests go to all of these. Since all 4 IPs are valid local addresses, all 4 interfaces are going to try to respond.

1) install arptables. Add this someplace during boot (/etc/rc.local here):

arptables -F INPUT
arptables -A INPUT -i eth0 --destination-ip ! 10.10.10.2 -j DROP
arptables -A INPUT -i eth1 --destination-ip ! 10.10.10.3 -j DROP
arptables -A INPUT -i eth2 --destination-ip ! 10.10.10.4 -j DROP
arptables -A INPUT -i eth3 --destination-ip ! 10.10.10.5 -j DROP

This will prevent broadcasts from going into the wrong interface. So, the correct interface will now be the only responder.

That by itself is not enough. The next bit is an ARP table problem. The requesting PC probably already has an ARP table entry, and so Linux it's going to use the interface associated with that. Until that ARP table entry expires, it's going to be trying to send ARP responses using that entry's interface, not the one associated with the ARP request.

The sysctl option *rp_filter* appears to be rejecting outgoing ARP response packets if they're on the wrong interface. So...

2) Disable *rp_filter*.

On Debian/Ubuntu, this means commenting out the two *rp_filter* lines in /etc/sysctl.d/10-network-security.conf.

This option was enabled for a reason... namely to help prevent cross-interface spoofing attacks. I read it verifies that the packet is legal for the interface it is coming in or going out on (by swapping MACs and IPs and seeing if it still routes over the same interface). So, normally it would be a bad idea to turn it off. In my case, all interfaces are on the same network... so that check doesn't matter really at all.

If I do add another interface and need the spoofing protection, perhaps can craft some arptables/iptables entries to do the same thing.

share|improve this answer

This has to do with the way Linux handles IPs and NICs. Basically, it treats an IP address as if it belongs to the box, and not just the specific NIC. The result is that you can get ARP responses from IP addresses on interfaces you wouldn't expect.

The solution is a sysctl option. As I recall, what you're looking for is:

net.ipv4.conf.default.arp_filter=1
net.ipv4.conf.all.arp_filter=1

That will fix the problem for you. Just add those to /etc/sysctl.conf and run 'sysctl -p' (or run each line as the argument to 'sysctl -w'.

This will cause Linux to only respond to ARP requests on the interface that an IP address is actually assigned to.

share|improve this answer
    
hmm... unfortunately it doesn't appear to work. I've pasted that into sysctl.conf, ran sysctl, even rebooted, no change. I can cat the options in /proc and see they're set, but if I arping from another box, all responses come from the same MAC. The interfaces are all on the same network (same broadcast, etc)... –  darron Jun 9 '09 at 1:00

The accepted answer in combination with this: http://www.linuxquestions.org/questions/linux-networking-3/multiple-interfaces-all-traffic-flows-through-just-one-538701/

where static routes are used to communicate with only desired IPs over certain interfaces, created a powerful and simple solution to a similar problem I had.

share|improve this answer

Can you bridge the gateway and have the firewall handle the IPs?

share|improve this answer
    
as far as I understand it, no... but someone please correct me if I'm wrong. –  darron Jul 30 '09 at 21:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.