3

On my office network I have two internet connections and one CentOS server running a website (HTTPS on port 443). The website should be publicly accessible through the public IP of the first internet connection (ISP-1). The other internet connection, ISP-2, id the default gateway on the network. Both internet connections have routers (the household-kind) with NAT, SPI firewalls etc. The router on ISP-2 is a Netgear WNDR3700 (aka N600) with original firmware.

The problem is that the website is unreachable. Looks like incoming traffic on ISP-1 will reach the server but the returning traffic is routed through ISP-2, effectively making the site unreachable. As far as I can tell I can't do port based routing on the WNDR3700.

What are my options to make this work? I've been looking at implementing an iptables / routing based solution on the server itself but haven't been able to make that work.

Update: Note that the server has one network interface connecting it to both routers.

2

5 Answers 5

3

If I am understanding your intentions correctly, you want your webserver to normally use ISP-2 as its default gateway for outgoing traffic, with the exception of its responses to external web requests, which must transit via ISP-1 instead. Here is a sketch of a solution using policy routing:

echo "101 webtraffic" >> /etc/iproute2/rt_tables

ip route add default table webtraffic via $ISP1_GW_LAN_IP
ip rule add fwmark 1 table webtraffic
iptables -t mangle -A OUTPUT -d \! $LAN_NET_PREFIX \
                             -p tcp -m tcp --sport 443 \
                             -j MARK --set-mark 1

where:

  • LAN_NET_PREFIX is your LAN's network prefix (e.g. 192.168.100.0/24), and
  • ISP1_GW_LAN_IP is the LAN IP address of your gateway to ISP-1 (e.g. 192.168.100.100).

The first ip command sets the default route on the webtraffic table to your ISP-1 gateway, and the second ensures that packets marked 1 are routed using the webtraffic table. Finally, the iptables rule marks the appropriate outgoing packets, ensuring that their next hop will be towards ISP-1.

Here is an alternate solution that uses an experimental iptables module, the ROUTE target:

iptables -t mangle -A POSTROUTING -d \! $LAN_NET_PREFIX \
                                  -p tcp -m tcp --sport 443 \
                                  -j ROUTE --gw $ISP1_GW_LAN_IP

This rule would override the routing decision for outgoing web response packets, sending them to your ISP-1 gateway, instead of to the default ISP-2. All other traffic, including web responses to clients on your LAN, would not be affected. As has been pointed out in the comments, the ROUTE target is very likely not to be implemented on any system that has not explicitly patched it into the kernel, since it is experimental.

7
  • Yes, that is exactly what I intended. Unfortunately the iptables version on the server doesn't support the ROUTE target... Any thoughts? Jan 21, 2011 at 21:39
  • @Marnix van Valen: Try using MARK and policy routing instead. I have updated my answer with an example. Jan 22, 2011 at 6:23
  • ROUTE was never part of the Linux kernel, nor has been endorsed as an out-of-tree module (-- policy routing is the right way). As such, it would only apply to a very customized subset of installations.
    – user61188
    Jan 24, 2011 at 1:56
  • @user61188: Thanks for pointing this out. I have updated my answer accordingly. Jan 24, 2011 at 2:39
  • @Steven Thanks, that worked! I needed to replace -t tcp with -p tcp in the iptables command though. Jan 25, 2011 at 11:03
5

I had the same issue, but I solved only with iproute2 (source routing). Marking with iptables wasn't necessary:

echo "101 webtraffic" >> /etc/iproute2/rt_tables
ip route add default table webtraffic via $ISP1_GW_LAN_IP
ip rule add from $ISP1_IP table webtraffic

The point is that not only web may use the ISP1 connection. You may choose. It's good because you may connect to the server from ssh from both connections if someone fails. As a CentOS user I created the following files so my changes weren't lost after reboot:

echo "default table webtraffic via $ISP1_GW_LAN_IP" >> /etc/sysconfig/network-scripts/route-eth1
echo "from $ISP1_IP table webtraffic" >> /etc/sysconfig/network-scripts/rule-eth1
0

This is called asynchronous routing. You have to point the default gateway of the webserver to the IP address of the router on ISP-1. If the clients that are accessing the web server came from the same IP you can route this without changing the default gateway, or, you can implement NAT on the ISP1 gateway to act like a reverse proxy and then route it at the web server. Regards.

1
  • I need ISP2 to be the default gateway on the server, except for web traffic. Changing the default gateway is not an option. ISP1 runs DD-WRT I'm not sure it'll do NAT on the inside, but I get the idea. Jan 21, 2011 at 21:46
0

Firewall people do inside nat to get over this problem. The inbound connection is natted to the firewall inside interface address and therefore does not have a routing problem at the server.

If the both inbound connections end up in same interface in same IP you have very limited (none?) options. See if you can get another IP for the same computer/interface and have one inbound connections in one IP and anoter inbound in another IP. After this you can do source routing easily.

If you can't get different sources connect to different IP addresses you can use iptables mac to redirect packets depending on MAC they were coming from.

   [!] --mac-source address
          Match  source  MAC  address.    It   must   be   of   the   form
          XX:XX:XX:XX:XX:XX.   Note that this only makes sense for packets
          coming from an Ethernet device and entering the PREROUTING, FOR‐
          WARD or INPUT chains.

Hope this helps.

1
  • Inside nat is not an option. The router won't be able to handle that. Interesting idea to route based on the mac. Hadn't thought of that. I'll definitely try to assign multiple IPs to the network interface, that would make things a lot simpler. Jan 21, 2011 at 22:03
0

Assuming your Webserver is on the same subnet as your workstations, why not just setup split DNS so that on the inside, yourwebsite.com resolves to it's internal IP address? Sure beats complicated asynchronous routing.

Also, investing a router than can support multiple WAN connections would make life a lot easier for you (and allow you do load balancing/failover between both Internet connections).

1
  • The server and workstations are on the same subnet, but I don't really see how a split DNS setup would help here. The problem is with people trying to access the server from the outside. I agree that a router with muliple WAN connections would help, but the way things are there is no budget for it. Besides, if a couple of lines of firewall config on the webserver will do the trick, why spend hundreds of euro's on hardware? Jan 25, 2011 at 10:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .