17

We've got AT&T U-Verse internet service, which has an extremely boneheaded DSL gateway.

We have 5 IPs (netmask 248), but the gateway is incapable of doing anything other than a single IP -> single MAC address mapping.

We have a single firewall machine, and we redirect different IP/port combos to different places inside a DMZ.

Our solution so far is to have a VMWare virtual machine on the firewall with 4 additional NICs in it, to get the other 4 IP addresses... however we have a problem.

The gateway is basically doing an ARP ping to see if the IP is responding on the expected MAC. With 4 NICs all on the same LAN, linux is responding to ARP requests for ALL IPs using a single interface. That's not what the gateway is expecting, and it's messing up the 3 other NICs. The gateway refuses to route incoming traffic for IPs where the ARP ping results aren't the expected MAC.

How can we get the ARP replies for eth0's IP to go out eth0, eth1's IP to go out eth1, etc?

EDIT

Christopher Cashell's response does not work in this situation. I had great hopes reading it, but... nope.

EDIT 2

Solved! See my answer below.

3
  • PS - no 'drop uverse' comments... U-Verse is 18mbps, vs. everything else's 3mbps or a very unreliable 10mbps via cable
    – darron
    Jun 8, 2009 at 22:46
  • @dblack: You forgot to add your answer :) Jun 9, 2009 at 4:56
  • The api.recaptcha.net site was down for a little while. The answer is up now.
    – darron
    Jun 9, 2009 at 5:36

5 Answers 5

15

Your chosen solution works, but there are alternatives that don't involve arptables. (Christopher Cashell was on the right track, originally, but he was off by a smidge.)

In short, you want to set these parameters:

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

These should be available when running a modern, 2.6 series Linux kernel. Check and make sure you that '/proc/sys/net/ipv4/conf//arp_announce' and /proc/sys/net/ipv4/conf//arp_ignore' exist on your system.

The 'arp_filter' parameter only works when your various IP addresses share a LAN segment but use different IP subnet. If they share the IP subnet, too, you need to use 'arp_ignore' and 'arp_announce', as above.

(I believe you may also need to set 'arp_filter' back to '0', too.)

2
  • What happened to arp_filter? It was an early (2003?) solution to the ARP problem, but it doesn't seem to work as described, at least on Centos 5. arp_ignore and arp_announce seem to be good.
    – pcapademic
    May 22, 2010 at 14:49
  • This solution, as is, appears to works only at the ARP level. Without additional route settings, the firewall might still elect the wrong outgoing card (and MAC) for its own outgoing IP traffic, but will (still) always receive the incoming IP traffic on the right card, leading to asymmetric path and rp_filter considerations.
    – A.B
    Mar 15, 2018 at 8:08
10

Okay, here's the solution. First, a recap:

Here's my basic network plan:

 eth0 10.10.10.2 netmask 255.255.255.248
 eth1 10.10.10.3 netmask 255.255.255.248
 eth2 10.10.10.4 netmask 255.255.255.248
 eth3 10.10.10.5 netmask 255.255.255.248

All the interfaces overlap. This is technically wrong, and the source of all my woes... but I have to do it because of this dumb residential gateway.

First, broadcast ARP requests go to all of these. Since all 4 IPs are valid local addresses, all 4 interfaces are going to try to respond.

1) install arptables. Add this someplace during boot (/etc/rc.local here):

arptables -F INPUT
arptables -A INPUT -i eth0 --destination-ip ! 10.10.10.2 -j DROP
arptables -A INPUT -i eth1 --destination-ip ! 10.10.10.3 -j DROP
arptables -A INPUT -i eth2 --destination-ip ! 10.10.10.4 -j DROP
arptables -A INPUT -i eth3 --destination-ip ! 10.10.10.5 -j DROP

This will prevent broadcasts from going into the wrong interface. So, the correct interface will now be the only responder.

That by itself is not enough. The next bit is an ARP table problem. The requesting PC probably already has an ARP table entry, and so Linux it's going to use the interface associated with that. Until that ARP table entry expires, it's going to be trying to send ARP responses using that entry's interface, not the one associated with the ARP request.

The sysctl option rp_filter appears to be rejecting outgoing ARP response packets if they're on the wrong interface. So...

2) Disable rp_filter.

On Debian/Ubuntu, this means commenting out the two rp_filter lines in /etc/sysctl.d/10-network-security.conf.

This option was enabled for a reason... namely to help prevent cross-interface spoofing attacks. I read it verifies that the packet is legal for the interface it is coming in or going out on (by swapping MACs and IPs and seeing if it still routes over the same interface). So, normally it would be a bad idea to turn it off. In my case, all interfaces are on the same network... so that check doesn't matter really at all.

If I do add another interface and need the spoofing protection, perhaps can craft some arptables/iptables entries to do the same thing.

5

This has to do with the way Linux handles IPs and NICs. Basically, it treats an IP address as if it belongs to the box, and not just the specific NIC. The result is that you can get ARP responses from IP addresses on interfaces you wouldn't expect.

The solution is a sysctl option. As I recall, what you're looking for is:

net.ipv4.conf.default.arp_filter=1
net.ipv4.conf.all.arp_filter=1

That will fix the problem for you. Just add those to /etc/sysctl.conf and run 'sysctl -p' (or run each line as the argument to 'sysctl -w'.

This will cause Linux to only respond to ARP requests on the interface that an IP address is actually assigned to.

2
  • hmm... unfortunately it doesn't appear to work. I've pasted that into sysctl.conf, ran sysctl, even rebooted, no change. I can cat the options in /proc and see they're set, but if I arping from another box, all responses come from the same MAC. The interfaces are all on the same network (same broadcast, etc)...
    – darron
    Jun 9, 2009 at 1:00
  • 1
    This is the correct solution if the interfaces are on the same network, but have addresses in different subnets. Confirmed working. Apr 7, 2017 at 8:45
1

The accepted answer in combination with this: http://www.linuxquestions.org/questions/linux-networking-3/multiple-interfaces-all-traffic-flows-through-just-one-538701/

where static routes are used to communicate with only desired IPs over certain interfaces, created a powerful and simple solution to a similar problem I had.

0

Can you bridge the gateway and have the firewall handle the IPs?

1
  • as far as I understand it, no... but someone please correct me if I'm wrong.
    – darron
    Jul 30, 2009 at 21:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .