Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have 2 internet links using 2 ADSL routers and I need to give access to the Internet for the network.

I have to route outgoing traffic based on port number, protocol, … using iproute2 and iptables on a linux router.

This is my network:

     (ISP-1)                              (ISP-2)
Dynamic public IP                    Dynamic public IP 
        |                                    |
+---------------+                    +---------------+
|ADSL Router (1)|                    |ADSL Router (2)|
+---------------+                    +---------------+
        |                                    |                
        |                                    |
        |                                    |
        |                                    |
        |        +------------------+        |
        |        |                  |        | --|eth1          eth2|--
                 |                  |
                 |   Linux Router   |
                 |                  |
                 |       eth0       |
                    Local Network:

I use the following script to setup the network configuration on the Linux router:


echo 1 >| /proc/sys/net/ipv4/ip_forward
echo 0 >| /proc/sys/net/ipv4/conf/all/rp_filter

# flush all iptables entries
iptables -t filter -F
iptables -t filter -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -t filter -P INPUT ACCEPT
iptables -t filter -P OUTPUT ACCEPT
iptables -t filter -P FORWARD ACCEPT

# marking packets
iptables -t mangle -A PREROUTING -i eth0 -p icmp -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -i eth0 -p udp  -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -i eth0 -p tcp  -j MARK --set-mark 2

# create routing tables and default routes
echo '1     ISP1' >> /etc/iproute2/rt_tables
echo '2     ISP2' >> /etc/iproute2/rt_tables
ip route add default via dev eth1 table ISP1
ip route add default via dev eth2 table ISP2

# routing based on previous marks
ip rule add fwmark 1 table ISP1
ip rule add fwmark 2 table ISP2

ip route flush cache

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE

The problem is that I can't connect to the internet from the network.

When I ping from this network to a remote server I can see (using Wireshark) replies returning back from the remote server to eth1 of the Linux router, but they don't reach out eth0.

Please help. And Thanks in advance.


I tried troubleshooting this weird issue for a week.

Troubleshooting commands output:

ip rule
0:  from all lookup local 
32764:  from all fwmark 0x1 lookup ISP1
32765:  from all fwmark 0x2 lookup ISP2 
32766:  from all lookup main 
32767:  from all lookup default 

ip route show table ISP1
default via dev eth1

ip route show table ISP2
default via dev eth2

ip route show table main dev eth2  proto kernel  scope link  src dev eth1  proto kernel  scope link  src dev eth0  proto kernel  scope link  src dev eth0  scope link  metric 1000 
default via dev eth1  metric 100

I can solve this problem partially by typing the following commands:

ip rule del fwmark 1 table ISP1
ip rule del fwmark 2 table ISP2
ip rule add from table ISP1

So I get all traffic from local network routed correctly trough ISP1 link and all PCs get Internet access.

But i am interested in routing based on packets marks.

share|improve this question
up vote 10 down vote accepted

After a lot of hard work I finally found what was the problem.

In fact it is not a routing problem, the script is correct but something is missing.

This command is not enough to disable rp_filter:

echo 0 >| /proc/sys/net/ipv4/conf/all/rp_filter

So the traffic returning back from Internet was drop at eth1 and eth2.

When I disabled rp_filter explicitly for both interfaces by adding the following commands:

echo 0 >| /proc/sys/net/ipv4/conf/eth1/rp_filter
echo 0 >| /proc/sys/net/ipv4/conf/eth2/rp_filter

The problem was solved and I get everything working perfectly.

The proof that Linux tutorials and documentation are not always completes.

share|improve this answer
The LARTC Howto does mention rp_filter, but it is a pretty obscure reference as a footnote. – Zoredache Aug 5 '11 at 19:34
it could be written as echo 0 >| /proc/sys/net/ipv4/conf/*/rp_filter – Sgaduuw Aug 6 '11 at 9:12
Can it? How can you pipe to more than one file? You can do for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 >| $f ; done although quickest in the simplest cases is echo 0 >/proc/sys/net/ipv4/conf/default/rp_filter – Cedric Knight Feb 22 '15 at 17:58

If the traffic's getting back to the router but not going out again, then the problem's in your routing table, some iptables rule, or you're not reading the wireshark correctly. There's not a lot of options for what would be stopping packets from exiting a system.

share|improve this answer
Thank you, but I am sure that traffic goes to Internet and return to eth1. – yozapho yozapho Aug 3 '11 at 10:42
Also Iptables is flushed – yozapho yozapho Aug 3 '11 at 10:42
So the problem's in your routing table, by a process of elimination. Problem solved. – womble Aug 3 '11 at 10:49

please look. seems looks like your issue.

share|improve this answer
We generally prefer people include more information than just a link, because of link rot. – Katherine Villyard Apr 15 '14 at 2:54

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.