Can't reach multi-homed Ubuntu from different subnet

Question

I have a multi-homed Ubuntu 12.04 server. I have two network interfaces connected to two different IP ranges.

2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet 190.113.X.X/29 brd 190.113.98.183 scope global eth1
(...)

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet 10.100.100.21/24 brd 10.100.100.255 scope global eth0
(...)

Whenever I try to access the server on the eth0 link from another subnet not pertaining to the 10.100.100.X network I get no response. I have iptables running in the server (given it has a public IP in eth1) but I allow all traffic from the private network on the eth0 link.

If I do a tcpdump on the interface in the server I have this (my pc is 10.100.102.22):

18:30:23.813889 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.100.102.22 tell 10.100.100.21, length 28
18:30:24.810691 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.100.102.22 tell 10.100.100.21, length 28
18:30:25.810718 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.100.102.22 tell 10.100.100.21, length 28

I can ping from the server to my PC:

PING 10.100.102.22 (10.100.102.22) 56(84) bytes of data.
64 bytes from 10.100.102.22: icmp_req=1 ttl=63 time=0.273 ms
64 bytes from 10.100.102.22: icmp_req=2 ttl=63 time=0.324 ms

The routing table is as follows: Table 1:

default via 10.100.100.1 dev eth0 
10.0.0.0/8 dev eth0  scope link  src 10.100.100.21

Table 2:

default via 190.113.X.X dev eth1 
190.113.X.X/29 dev eth1  scope link  src 190.113.X.X

Default:

default via 10.100.100.1 dev eth0  metric 100 
10.100.100.0/24 dev eth0  proto kernel  scope link  src 10.100.100.21 
190.113.X.X/29 dev eth1  proto kernel  scope link  src 190.113.X.X 

Answer 1

The client needs to know the route to the 10.100.100.0/24 network as well.

so you either need to add a route on the client

ip r a 10.100.100.0/24 via 10.100.100.1 dev eth0

or you need to add the route on the default gateway the client is using.

you can think of it this way: the client needs to know how to reach 10.100.100.21, if it doesnt, it will try the default gateway, if the gateway doesn't know, you are out of luck.