How do i modify route tables in EC2 instance to send traffic via eth1?

Question

I have an ec2 AmazonLinux2 instance. It has a primary nic on eth0. I went ahead and attached another eni (with an associated public ip) eth1. I would like to make sure that I can send traffic via the eth1 as well but unable to.

curl --interface eth0 ifconfig.me --> Works, returns the public ip of the instance
curl --interface eth1 ifconfig.me --> Does not work, the call just hangs

Here are my interfaces

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 02:82:39:f5:b2:61 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.156/23 brd 192.168.1.255 scope global dynamic eth0
       valid_lft 2293sec preferred_lft 2293sec
    inet6 fe80::82:39ff:fef5:b261/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 02:85:86:84:a8:1b brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.8/23 brd 192.168.1.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::85:86ff:fe84:a81b/64 scope link
       valid_lft forever preferred_lft forever

Route Table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
169.254.169.254 0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.0.0     0.0.0.0         255.255.254.0   U     0      0        0 eth0

Here are my steps that i took to add static route

1. echo 2 mytable >> /etc/iproute2/rt_tables
2. sudo ip route add default via 192.168.0.1 dev eth1 table mytable
3. sudo ip rule add from 192.168.0.8 lookup mytable prio 1000
4. ip route flush table cache

I have seen various posts here which highlights different ways of doing the same, i have tried them in vain. Can someone help me with whats going on here

These steps are inspired from the post http://www.rjsystems.nl/en/2100-adv-routing.php.

Thanks Kay

Answer 1

You have two routes for the same subnet, that's why its not working. You need your route rules to concern stricktly different subnets.

This configuration forward all request in destination of all the ip from 192.168.0.1 to 192.168.1.255 through eth0 :

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
192.168.0.0     0.0.0.0         255.255.254.0   U     0      0        0 eth0

Even when adding your rules for eth1, as it is defined for the same subnet, your rules will be ignored.

You should have two rules with two different subnets, something like that :

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     192.168.0.1     255.255.255.0   U     0      0        0 eth0
192.168.0.0     X.X.X.X         255.255.255.0   U     0      0        0 eth1

Try changing your mask :)

Answer 2

Following steps worked for me.

ip route add <CIDR Range of Subnet> dev eth1 table 2
ip rule add from <IP of eth1> table 2
ip route add default via <Gateway of subnet> dev eth1 table 2
ip route flush cache

Hope that helps someone