1

I have severs behind two different public IPs and doing DNAT to send the traffic to different internal servers depending on the port.


-------------------------                  -----------------------
| server a (port 80/tcp)|                  |       router A      |
|   (eth0) 192.168.1.123|..................|192.168.1.1 (eth1)   |
-------------------------         :        |       (eth0) 1.2.3.4|............
                                  :        -----------------------           :
                                  :                                          :
                                  :        -----------------------           :.... INTERNET
-------------------------         :        |       router B      |           :
| server a (port 25/tcp)|         :        |       (eth0) 2.3.4.5|...........:
|   (eth0) 192.168.1.234|..................|192.168.1.2 (eth1)   |
-------------------------                  -----------------------

On router A and B I enabled the forwarding and the following iptables:

iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -o eth0 -j MASQUERADE
iptables -t nat -A PREROUTING -d ${externalIP} -i eth0 -p tcp --dport 25 -j DNAT --to 192.168.1.234
iptables -t nat -A PREROUTING -d ${externalIP} -i eth0 -p tcp --dport 80 -j DNAT --to 192.168.1.123

Actually I have the following route added on the servers A and B:

route add default gw 192.168.1.1

So all the internal traffic to internet goes through router A and all the traffic to port 25 and 80 are correctly sent to server A and B, but the connections only works if the traffic came through router A. Indeed, the traffic through router B arrive to servers but it is returned through router A instead B, so the connections does not work.

I added a new route on the servers:

route add default fw 192.168.1.2

route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
0.0.0.0         192.168.1.2     0.0.0.0         UG    0      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0

But the connections are still not working.

How can I manage that?

It is not important to balance the traffic Internal -> External, but I want that all the incoming connections from internet against the external IPs are correctly established.

NOTE: I also tried to do the DNAT on one of the routers against the external IP of the other router, but they are on different networks and it don't worked:

iptables -t nat -I PREROUTING -p tcp --dport 25 -d ${ExternalIPA} -i eth0 -j DNAT --to ${ExternalIPB}
0

Your current setup will not work as intended. You need to have a virtual IP that will float over Router-A and Router-B. This will also load balance between the two routers with simplicity. In this example, I will use the industry standard VRRP on a Linux router (Ubuntu-16).

Here is the procedure:

On both Linux Router-A and Router-B, execute the following commands:

$ sudo echo "net.ipv4.ip_nonlocal_bind=1" >> /etc/sysctl.conf
$ sudo sysctl -p

Install keepalived on both routers

$ sudo apt-get update
$ sudo apt-get install keepalived -y

Then go to Router-A that will act as MASTER. Create a new configuration file /etc/keepalived/keepalived.conf with following entries:

vrrp_instance VI_1 {
    interface eth1
    state MASTER
    virtual_router_id 50
    priority 101

    authentication {
        auth_type AH
        auth_pass pass123
    }

    virtual_ipaddress {
        192.168.1.99
    }
}

Then go to Router-B that will act as BACKUP, and do the same to the new configuration file /etc/keepalived/keepalived.conf:

vrrp_instance VI_1 {
    interface eth1
    state BACKUP
    virtual_router_id 50
    priority 100

    authentication {
        auth_type AH
        auth_pass pass123
    }

    virtual_ipaddress {
        192.168.1.99
    }
}

Explanation:

net.ipv4.ip_nonlocal_bind=1 tells the kernel that the virtual IP must NOT be bound to any physical interface.

vrrp_instance VI_1 must match on both routers. This is an identifier since you can have multiple VRRPs running on the same router.

interface eth1 is the interface that is LAN facing (facing the internal servers).

state MASTER on Router-A and state BACKUP on Router-B is self-explanatory.

virtual_router_id 50 must match on both routers. This is an identifier.

priority 101 on Router-A and priority 100 on Router-B (Router-A has a higher priority in this case).

authentication must match on both routers. In this case I used a pre-shared key of passw123

virtual_ipaddress is the floating IP address that VRRP will use. This IP must match on both routers. This IP must not be used by any device on the LAN segment. In this case I chose 192.168.1.99

Enable keepalived service

$ sudo systemctl enable keepalived

Start keepalived service

$ sudo systemctl start keepalived

Time to manipulate iptables rules on both routers.

Clear all NAT rules from both routers (get rid of broken NAT rules)

$ sudo iptables -t nat -F
$ sudo iptables -t mangle -F

Then add correct NAT rules:

Router-A

$ sudo iptables -t nat -A POSTROUTING -o eth0 -s 192.168.1.0/24 -j SNAT --to 192.168.1.99
$ sudo iptables -t nat -A PREROUTING -d ${externalIP} -i eth0 -p tcp --dport 25 -j DNAT --to 192.168.1.234

Router-B

$ sudo iptables -t nat -A POSTROUTING -o eth0 -s 192.168.1.0/24 -j SNAT --to 192.168.1.99
$ sudo iptables -t nat -A PREROUTING -d ${externalIP} -i eth0 -p tcp --dport 80 -j DNAT --to 192.168.1.123

Thats all you need from the router side. Now hop over to the servers (Server-A and Server-B), delete the existing default gateway (192.168.1.1) and assign the new gateway (192.168.1.99)

$ sudo ip route del 0/0
$ sudo route add default gw 192.168.1.99

When connections are establised from internet, the two servers (Server-A and Server-B) will return traffic to the virtual IP (192.168.1.99) that is floating above Router-A and Router-B

Reference: https://www.keepalived.org/manpage.html

2
  • 1
    Yes, VRRP will fix the servers choosing a default route, but that does not solve the asymmetric routing problem with NAT. When a connection comes into one of the routers, the reply must go back out the same router. The routers do not share NAT tables and have different WAN addresses to which the LAN addresses with translate. IF traffic goes out the master (Router A), but came into the backup (Router B), you will still have the original problem. – Ron Maupin Dec 12 '20 at 23:30
  • Please accept my apology. I have fixed the NAT rules. I have tested and its working 100%. – Bruce Malaudzi Dec 13 '20 at 16:12
0

Finally I managed to get it working without using a virtual IP, basically using CONNNMARK and adding different routes:

NOTE:

  • Router A eth1 MAC = AA:AA:AA:AA:AA:AA
  • Router B eth1 MAC = BB:BB:BB:BB:BB:BB

I have to add following routes and iptables on all the servers that are using asymetric routes:

# add the two routing tables
ip route add to default table 11 via 192.168.1.1 dev eth0
ip route add to default table 33 via 192.168.1.2 dev eth0

# add the mark to the routing tables
ip rule add priority 99 table 11 fwmark 11
ip rule add priority 99 table 33 fwmark 33

# mark the packets that came from the different routes
iptables -t mangle -A OUTPUT ! -d 192.168.1.0/24 -m addrtype --dst-type UNICAST -j CONNMARK --restore-mark
iptables -A INPUT -i eth0 ! -s 192.168.1.0/24 -m addrtype --src-type UNICAST -m mac --mac-source AA:AA:AA:AA:AA:AA -j CONNMARK --set-mark 33
iptables -A INPUT -i eth0 ! -s 192.168.1.0/24 -m addrtype --src-type UNICAST -m mac --mac-source BB:BB:BB:BB:BB:BB -j CONNMARK --set-mark 11

Now everything is working fine. All the connections from internet to any of the external IP addressees are routed through the router that manage the DNAT connection.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.