0

We've just implemented a secondary internet connection. It will be used as pure failover so for the time being I just want to avoid async routing issues.

I've setup 2 routing tables:

fw1 ~ # ip route sho tabl 10 | grep default
default via 59.167.xx.xx dev eth0  src 59.167.xx.xx
fw1 ~ # ip route sho tabl 20 | grep default
default via 115.64.xx.xx dev eth0.2  src 115.64.xx.xx

Table 10 is our primary connection, Table 20 is the new connection. I've setup the rules to select the routing table based on the connection mark:

fw1 ~ # ip rule sho
0:  from all lookup local 
32736:  from all fwmark 0x14 lookup 20 
32737:  from all fwmark 0xa lookup 10 
32766:  from all lookup main 
32767:  from all lookup default 

The appropriate rules are also setup in iptables:

fw1 ~ # iptables -t mangle -nvL PREROUTING
Chain PREROUTING (policy ACCEPT 300K packets, 164M bytes)
 pkts bytes target     prot opt in     out     source               destination         
 372K  201M CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0           CONNMARK restore 
 371K  201M PBR        all  --  *      *       0.0.0.0/0            0.0.0.0/0           /* this stream has no mark; send it to the PBR chain */ mark match 0x0 
fw1 ~ # iptables -t mangle -nvL PBR
Chain PBR (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   10   666 MARK-gw1   all  --  eth0   *       0.0.0.0/0            59.167.xx.xx/29    /* prevent asynchronous routing */ state NEW mark match 0x0 
   18  1128 MARK-gw2   all  --  eth0.2 *       0.0.0.0/0            115.64.xx.xx/29    /* prevent asynchronous routing */ state NEW mark match 0x0 
fw1 ~ # iptables -t mangle -nvL MARK-gw1
Chain MARK-gw1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   10   666 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0           /* send via 59.167.xx.xx */ MARK set 0xa 
   10   666 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0           CONNMARK save 
   10   666 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           
fw1 ~ # iptables -t mangle -nvL MARK-gw2
Chain MARK-gw2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   18  1128 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0           /* send via 115.64.xx.xx */ MARK set 0x14 
   18  1128 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0           CONNMARK save 
   18  1128 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

I'm testing with a ping from an outside VPS. The connection tracking table confirms that the mark is being set correctly:

icmp     1 28 src=173.255.xx.xx dst=115.64.xx.xx type=8 code=0 id=29301 src=115.64.xx.xx dst=173.255.xx.xx type=0 code=0 id=29301 mark=20 secmark=0 use=2

But tcpdump shows the reply being sent via eth0 instead of eth0.2 even though it has the correct source address:

fw1 ~ # tcpdump -lnn -i eth0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
14:07:56.429765 IP 115.64.xx.xx > 173.255.xx.xx: ICMP echo reply, id 29301, seq 619, length 64
0

Well I don't know exactly why mine didn't work, but I reimplemented it using the 'throw default' option as per this [1] guide. I know I shouldn't link to an external site as an answer but it's way too long to copy and paste here I'm sorry.

[1] http://www.cyber.com.au/~twb/doc/dual-uplink.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.