In my environment most of the servers have 2 network cards. A primary/user network card used by the application, and a secondary/backup adapter used for backups, bulk copies etc.
I just set up a new server and have a problem pinging its secondary adapter from a clients that only have a primary/user adapter.
From my test client I am able to ping my other servers on their primary/secondary adapters.
The Networks:
192.168.151.0 == user network with my servers
192.168.147.0 == user network with my client
192.168.245.0 == backup network
The Computers:
newServer (rhel 6) [ a.k.a. the troublemaker]
eth0 : 192.168.151.180 (user network)
eth1 : 192.168.245.187 (backup network)
oldServer (rhel 5)
eth0 : 192.168.151.203 (user network)
eth1 : 192.168.245.236 (backup network)
pubClient (rhel5) [client system with only a user interface]
- eth0 : 192.168.147.118 (user)
From pubClient I can ping oldServer on either its eth0 or eth1 ip addresses. I noticed (tcpdump) that when I ping eth1, the return path of the ping is the eth0 interface of oldServer.
From pubClient I can ping newServer's eth0 interface, but when I ping newServer's eth1 adapter nothing happens. Using tcpdump I see the traffic coming in to the backup network interface, but no traffic coming out. The eth1 adapter is working fine - from newServer I can ping other devices on the backup network using the 245 interface, or pubClient without any indication of a problem.
The routing tables on both servers are simple and functionally identical.
newServer # ip route list
192.168.151.0/24 dev eth0 proto kernel scope link src 192.168.151.180
192.168.245.0/24 dev eth1 proto kernel scope link src 192.168.245.187
default via 192.168.151.5 dev eth0
oldServer # ip route list
192.168.151.0/24 dev eth0 proto kernel scope link src 192.168.151.203
192.168.245.0/24 dev eth1 proto kernel scope link src 192.168.245.236
default via 192.168.151.5 dev eth2
There are no firewalls running on either box. net.ipv4.ip_forward = 0
on both servers. The newServer is running RHEL6 I wonder if a default
for some kind of network traversal (like ip_forward) has been disabled
by default?
My network guys insist nothing is wrong with the routing between these machines.
Any guesses to what the problem might be would be appreciated.
Cheers
-Scott