I have a Linux (Ubuntu 12.04) machine (c220-1) that has multiple interfaces. The two of interest here are:
- eth2: 10.10.0.131/24
- eth3: 10.20.0.2/24
The default route is set to 10.20.0.2. However, I'd like to be able to ssh into this machine by connecting to either 10.20.0.2 or 10.10.0.131, from a separate network (192.168.3.0/24)
To be able to ssh into 10.10.0.131 and avoid the symmetric routing problem, I used source routing policy:
root@c220-1:~# ip route list table eth2
default via 10.10.0.1 dev eth2
I added this rule by adding the following line to /etc/iproute2/rt_tables
12 eth2
And then I did
root@c220-1:~# ip route add default via 10.10.0.1 table eth2
root@c220-1:~# ip rule add from 10.10.0.131 lookup eth2
This works, I can ssh to either 10.10.0.131 or 10.20.0.2. But I can no longer connect to 10.10.0.131 from other machines on the 10.10.0.0/24 network. I can arping in both directions, and I can ping from c220-1 (10.10.0.131/24) to c220-2 (10.10.0.132/24). But I can't ping from c220-2 to c220-1.
If I do a tcpdump, I can confirm that c220-1 is receiving the ICMP requests and sending replies:
root@c220-1:~# tcpdump -i eth2 icmp -vvv -n -e
tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
23:15:11.455818 d4:8c:b5:4d:a5:3a > 60:73:5c:68:b4:b6, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
10.10.0.132 > 10.10.0.131: ICMP echo request, id 8805, seq 1, length 64
23:15:11.455877 60:73:5c:68:b4:b6 > 30:f7:0d:bc:bb:52, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 16273, offset 0, flags [none], proto ICMP (1), length 84)
10.10.0.131 > 10.10.0.132: ICMP echo reply, id 8805, seq 1, length 64
23:15:12.463394 d4:8c:b5:4d:a5:3a > 60:73:5c:68:b4:b6, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
10.10.0.132 > 10.10.0.131: ICMP echo request, id 8805, seq 2, length 64
23:15:12.463451 60:73:5c:68:b4:b6 > 30:f7:0d:bc:bb:52, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 16274, offset 0, flags [none], proto ICMP (1), length 84)
10.10.0.131 > 10.10.0.132: ICMP echo reply, id 8805, seq 2, length 64
You can see that the requests are coming from d4:8c:b5:4d:a5:3a
, but the replies are going to 30:f7:0d:bc:bb:52
.
As expected, the ICMP replies don't show up on c220-2:
root@c220-2:~# tcpdump -i eth2 icmp -vvv -n -e
tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
23:16:19.944225 d4:8c:b5:4d:a5:3a > 60:73:5c:68:b4:b6, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
10.10.0.132 > 10.10.0.131: ICMP echo request, id 9380, seq 1, length 64
23:16:20.952497 d4:8c:b5:4d:a5:3a > 60:73:5c:68:b4:b6, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
10.10.0.132 > 10.10.0.131: ICMP echo request, id 9380, seq 2, length 64
23:16:21.960458 d4:8c:b5:4d:a5:3a > 60:73:5c:68:b4:b6, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
10.10.0.132 > 10.10.0.131: ICMP echo request, id 9380, seq 3, length 64
The destination MAC address for the replies matches the MAC address of the gateway, 10.10.0.1:
root@c220-2:~# arping -i eth2 10.10.0.1
ARPING 10.10.0.1
60 bytes from 30:f7:0d:bc:bb:52 (10.10.0.1): index=0 time=214.100 usec
If I remove the source policy routing rules entirely, then I am able to connect from c220-2, but I can't connect to 10.10.0.131 from 192.168.3.1/24.
How do I specify the rules so that the source policy routing doesn't send packets to 10.10.0.1 when they should go to another machine on the local network?
route -n
) is fine. The packet should hit the longest match and go out via eth2. But you say you have source routing. If it is present that would take precedence. Can you show your source routing settings?