Routing + iptables? ( ELB + NAT Gateway )

Question

I have a group of identical amazon linux ec-2 frontend web servers, which serve their web content through Elastic Load Balancer (ELB).

Each of them has a public ip, so when they connect to an outside service in www, their ips are logged as several different ones.

I'd like these servers to keep serving their web content through ELB, but connect to the outside world through NAT Gateway using it's single elastic ip.

Here's what I did:

1) created the NAT Gateway in the same subnet (10.0.1.100 is it's ip) as ec2 machines + assigned an elastic ip to it. 2)

route add -host ifconfig.co gw 10.0.1.100 dev eth0
route add -host otherservice gw 10.0.1.100 dev eth0
...

Now curl ifconfig.co returns the external elastic ip of NAT Gateway & it's great :), however I need all traffic to go through the NAT Gateway, except the one coming through ELB.

UPDATE: Here's what I did to complete the task: 1) removed the custom route to ifconfig.co & other services;

2) created second subnet 10.0.200.0/24;

3) added NAT Gateway as default gateway into it's routing table;

4) added the following lines to /etc/rc.d/rc.local in order to run them on boot:

for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 > $f; done
echo 1 > /proc/sys/net/ipv4/route/flush
if [ -z "`cat /etc/iproute2/rt_tables | grep '^200'`" ] ; then echo "200 nat" >> /etc/iproute2/rt_tables; fi
ip route add default via 10.0.200.1 dev eth1 table nat
ip rule add fwmark 0x1 table nat
ip2addr=$(ip addr | grep 'state UP' -A2 | tail -n1 | awk '{print $2}' | cut -f1  -d'/')
iptables -A OUTPUT -t mangle -o eth0 -p tcp --dport 80 -j MARK --set-mark 1
iptables -A POSTROUTING -t nat -o eth1 -p tcp --dport 80 -j SNAT --to $ip2addr
iptables -A OUTPUT -t mangle -o eth0 -p tcp --dport 443 -j MARK --set-mark 1
iptables -A POSTROUTING -t nat -o eth1 -p tcp --dport 443 -j SNAT --to $ip2addr

Answer 1

The ELB sits on the same network as your web servers. From the perspective of the web servers, all web requests will appear to originate from the load balancer. Because the load balancer is on the same network no fancy routing is needed.

Simply add a single default route to your NAT gateway on each web server. Or, if you want all servers on the subnet to use the NAT gateway, change the default route on your subnet as suggested above.