The routing table by default will only route traffic to
Even though ubuntu detects the other ENI, you still have to route traffic to it.
You'll have to do some advanced routing:
1) Enabling access to 2nd ENI immediately and temporarily.
# this will show your route table, i'll assume you have eth0 and eth1
# and your default is for eth0 to point to the gateway
# for this example lets assume the following:
# eth0 = 192.168.100.5
# eth1 = 192.168.100.10
# gateway = 192.168.100.1
ip route show ;
# first step is to create a routing table for your new device
cat /etc/iproute2/rt_tables ;
echo 2 eth1_rt >> /etc/iproute2/rt_tables ;
# next add the eth1_rt route table, so by default it will also point to the gateway
ip route add default via 192.168.100.1 dev eth1 table eth1_rt ;
# next take a look at your ip rules
# i'll assume the defaults here, and things flows to default with priority 32767
# let's add a rule, if we see traffic from eth1's IP address,
# use its new routing table we setup, and give it higher priority than default
ip rule add from 192.168.100.10 lookup eth1_rt prio 1000 ;
# done! now check your traffic from both IPs, they should both work.
2) Enabling access to 2nd ENI on reboot but persistently.
Additionally, if you want this change to persist, you can make all these changes in the interface file and just restart the network service or reboot for it to take effect.
# NOTE: add the eth1_rt routing table to /etc/iproute2/rt_tables as show in previous section
# original config to make dchp, I add mine to /etc/network/interfaces.d/eth1.cfg
iface eth1 inet dchp
# your extra rules for eth1
up ip route add default via 192.168.100.1 dev eth1 table eth1_rt
up ip rule add from 192.168.100.10 lookup eth1_rt prio 1000
For this to take full effect, reboot the system.
NOTE: I tried
/etc/init.d/networking restart; but it didn't pick up the route/rule changes, not sure why, so I had reboot. In the event you want to make it immediate and persistent, do both methods.