Let me start with a crude network diagram.
Internal Layer 3 switch (Multiple VLANS, including 172.16.220.0 network) --> Internal network (172.16.220.0/24) --> eth1(172.16.220.100)-Ubuntu 10.04 VM-eth0(192.168.1.100) --> NATed network (192.168.1.0/24) --> ASA --> ISP.
Here's the problem I'm having on two different servers. I built the servers with only one interface, eth0, which is connected to the inside interface of the ASA. I then added another interface for the internal 172.16.220.0 network. I had a gateway set on eth0 and not on eth1 in /etc/network/interfaces. I could access the server externally, but not internally. If I removed the gateway from eth0 and set one for eth1, of course I could access it from internal, but not external.
I tried to set some policy based routing, so that any traffic from the inside network (multiple vlans - 172.16.0.0/16) would be routed back out eth1, but it did not work. Obviously I'm missing some piece of this puzzle, and have likely mis-configured these servers. Could someone help me get this to work so that I can access the server from both externally and internally. I'm getting frustrated at having to use a 3G connection to SSH into my servers to configure!!
Here's my /etc/network/interfaces config:
# This file describes the network interfaces available on your system
auto loopback network interface
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.100
netmask 255.255.255.0
# gateway 192.168.1.1
auto eth1
iface eth1 inet static
address 172.16.220.100
netmask 255.255.255.0
gateway 172.16.220.1
auto eth0
iface eth0 inet static
address 192.168.1.100
netmask 255.255.255.0
and my kernel routing table:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 * 255.255.255.0 U 0 0 0 eth0
172.16.220.0 * 255.255.255.0 U 0 0 0 eth1
default 172.16.220.1 0.0.0.0 UG 100 0 0 eth1
I had tried enabling IPv4 forwarding in /etc/sysctl.conf, but that didn't work either.
If anything else is needed, just let me know.
Thanks for your help.
EDIT - added kernel routing table after setting static route suggested by faker
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.16.220.0 172.16.220.1 255.255.255.0 UG 0 0 0 eth1
172.16.220.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 eth0