Configuring route to use the same interface for outbound traffic as that of inbound traffic

Question

I have a Linux box with two interfaces: one for data and the other for management purpose. If both are on the same network, I have a route added for both interfaces.

Imagine the routing table is:

192.168.132.0 255.255.255.0 eth0
192.168.132.0 255.255.255.0 eth1 ( management interface)

The problem is that even if a connection is initiated to mgmt IP (eth1), return traffic will be on eth0.

How can I ensure that return traffic on a connection uses the same interface as that of forwardeded traffic?

Answer 1

You can do this using a combination of iptables/netfilter with the conntrack and (conn-)mark modules, and policy-routing.

in the iptables MANGLE table:

-A PREROUTING          -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark
-A OUTPUT              -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark
-A PREROUTING -i eth0  -m state --state NEW                 -j CONNMARK --set-mark 1
-A PREROUTING -i eth1  -m state --state NEW                 -j CONNMARK --set-mark 2
-A PREROUTING -m connmark --mark 1                          -j MARK --set-mark 1
-A PREROUTING -m connmark --mark 2                          -j MARK --set-mark 2
-A PREROUTING -m state --state NEW -m connmark ! --mark 0   -j CONNMARK --save-mark

I assume you have a default route on eth0 so we only need to set up a "magic" route for those pakets coming in via eth1.

Using the iproute2 package:

ip rule add fwmark 2 table eth1
ip route add default via 192.168.132.1 table eth1

What this does is: Every connection coming in via eth1 will be marked by iptables with a "2", so every paket in that connection has a mark in it, that can be read outside the iptables code. Then a policy route is added, that routes every paket having a mark of "2" via the routing table "eth1" which has a default route to your router, but sends out the pakets via eth1.

Voila.

That setup above assumes you have the same IP on both interfaces. If you have different IPs on both, you can simplify the setup by only using a policy-route on the source ip (i assume 192.168.132.3 on eth1):

ip rule add from 192.168.132.3 table eth1
ip route add default via 192.168.132.3 table eth1

There is no need for connmarking via iptables then.

Answer 2

If you are trying to keep your management traffic separate from other traffic, you will not achieve this simply by having your management leaving a different port on your server. You need to keep the traffic separate after it leaves the server too - which you will not achieve by having it travel on the same subnet.

So to separate your traffic you really need to split your network up into multiple subnets, using separate switching, or VLANs, and use routing to route traffic between the different subnets. Then on your server you have a separate management interface that is only connected to the management subnet. Ideally you do your management tasks from hosts connected only to this subnet too.

That will enable you to really get what you want, and will be simpler to configure and understand - although not necessarily cheaper to setup. I'm assuming you have a clear business case for separating your management traffic.

If all you want to do is limit management tasks on your server to specific hosts on your network you would achieve this more simply by setting up iptables to only allow management type traffic (e.g. identified by port such as ssh) from those specific hosts IP addresses.