0

I have a group of identical amazon linux ec-2 frontend web servers, which serve their web content through Elastic Load Balancer (ELB).

Each of them has a public ip, so when they connect to an outside service in www, their ips are logged as several different ones.

I'd like these servers to keep serving their web content through ELB, but connect to the outside world through NAT Gateway using it's single elastic ip.

Here's what I did:

1) created the NAT Gateway in the same subnet (10.0.1.100 is it's ip) as ec2 machines + assigned an elastic ip to it. 2)

route add -host ifconfig.co gw 10.0.1.100 dev eth0
route add -host otherservice gw 10.0.1.100 dev eth0
...

Now curl ifconfig.co returns the external elastic ip of NAT Gateway & it's great :), however I need all traffic to go through the NAT Gateway, except the one coming through ELB.

UPDATE: Here's what I did to complete the task: 1) removed the custom route to ifconfig.co & other services;

2) created second subnet 10.0.200.0/24;

3) added NAT Gateway as default gateway into it's routing table;

4) added the following lines to /etc/rc.d/rc.local in order to run them on boot:

for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 > $f; done
echo 1 > /proc/sys/net/ipv4/route/flush
if [ -z "`cat /etc/iproute2/rt_tables | grep '^200'`" ] ; then echo "200 nat" >> /etc/iproute2/rt_tables; fi
ip route add default via 10.0.200.1 dev eth1 table nat
ip rule add fwmark 0x1 table nat
ip2addr=$(ip addr | grep 'state UP' -A2 | tail -n1 | awk '{print $2}' | cut -f1  -d'/')
iptables -A OUTPUT -t mangle -o eth0 -p tcp --dport 80 -j MARK --set-mark 1
iptables -A POSTROUTING -t nat -o eth1 -p tcp --dport 80 -j SNAT --to $ip2addr
iptables -A OUTPUT -t mangle -o eth0 -p tcp --dport 443 -j MARK --set-mark 1
iptables -A POSTROUTING -t nat -o eth1 -p tcp --dport 443 -j SNAT --to $ip2addr
4
  • Instead of adding a host route, edit your subnet routing table so that the default route points to the NAT gateway. No need to do anything else, the ELB will continue to function as desired. – EEAA Jul 17 '17 at 23:33
  • I've updated my original post with my solution. If you change the default route to NAT Gateway in the same subnet, hosts lose connectivity (tested). – GTXBxaKgCANmT9D9 Jul 18 '17 at 13:14
  • The NAT Gateway must be on a different subnet than any instances that it serves, otherwise its default route points back to itself, since its default route is derived from the subnet on which it is located... but you are doing this the hard way with two interfaces and iptables. There's no need for the instances to be on the public subnet with the ELB -- in fact, that's really not how it's intended to be set up. ELB and NAT device on public subnet, instances on private subnet, and everything just works because the web traffic is coming in with the ELB source IP, not the client IP. – Michael - sqlbot Jul 18 '17 at 15:46
  • So you're saying that I should remove 10.0.1.0/24 interfaces from the instances (the one you're calling public), leave only 10.0.200.0/24? Then NAT Gateway should be in the 10.0.1.0/24 network & then I should change the default gateway in the routing table for 10.0.200.0/24 to NAT Gateway? – GTXBxaKgCANmT9D9 Jul 24 '17 at 18:03
0

The ELB sits on the same network as your web servers. From the perspective of the web servers, all web requests will appear to originate from the load balancer. Because the load balancer is on the same network no fancy routing is needed.

Simply add a single default route to your NAT gateway on each web server. Or, if you want all servers on the subnet to use the NAT gateway, change the default route on your subnet as suggested above.

1
  • I've updated my original post with my solution. If you change the default route to NAT Gateway in the same subnet, hosts lose connectivity (tested). – GTXBxaKgCANmT9D9 Jul 18 '17 at 13:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.