0

I have a CentOS 5.5 server that is using 1 of the 2 on-board network ports (eth0) and 2 PCI NICs (eth2 & eth3) - each has it's own IP on a different subnet with different gateway on a different VLAN. The IPs on eth0 & eth2 are working fine, but the IP on eth3 is not responding to requests including ICMP, though it does show as active on the switch and if I ping the IP locally on the server it does work.

I am guessing this some kind of IP routing issue with Linux, maybe it's trying to respond to eth3 requests on eth2 or eth0 instead of eth3?

Hopefully someone has some suggestions for me please?

eth0: x.x.236.2 netmask 255.255.252.0 gateway x.x.236.1
eth2: x.x.232.2 netmask 255.255.252.0 gateway x.x.232.1
eth3: x.x.208.2 netmask 255.255.248.0 gateway x.x.208.1

Output from "ip route":

x.x.236.0/22 dev eth0  proto kernel  scope link  src x.x.236.2
x.x.232.0/22 dev eth2  proto kernel  scope link  src x.x.232.2
x.x.208.0/21 dev eth3  proto kernel  scope link  src x.x.208.2
169.254.0.0/16 dev eth3  scope link
default via x.x.232.1 dev eth2
default via x.x.236.1 dev eth0
default via x.x.208.1 dev eth3
1
  • disconnect the other two interfaces. Can you reach eth3?
    – Zoredache
    Oct 12, 2011 at 15:32

2 Answers 2

3

Linux sends outbound packets on the interface that is closest to the destination. It doesn't consider a packet a 'response' to another packet for routing purposes, nor would it make sense for it to do so.

You should not typically have three gateways, except for redundancy. If a router cannot handle a packet from any of the machine's IP addresses to any host that you don't have a more-specific route to, it should not be listed as a gateway.

There are no such thing as "eth3 requests". There are just packets, with source and destination IP addresses.

Three listed defaults means three machines that can handle any packet to a non-local host.

4
  • So if it's only 1 of the IPs that isn't working this suggests it's a problem with the router?
    – Tim
    Oct 12, 2011 at 13:52
  • 1
    It suggests that the router isn't correctly configured to handle packets with that source IP address. If it's supposed to, then the problem is with the router. If it's not supposed to, then the problem is that you have a special requirement but a standard configuration. The standard assumption is that a router can handle a packet with any source address so long as its destination is one the router is supposed to handle. Oct 12, 2011 at 15:24
  • 1
    Thank you the problem was found to be a faulty switch port it has now been changed to a different port and the server has been re-configured to use only 1 default gateway for all 3 NICs and everything is working perfectly.
    – Tim
    Oct 13, 2011 at 11:02
  • Wow. Sometimes when you hear hoofbeats behind you, it really is zebras. Oct 13, 2011 at 11:08
1

If you disable Reverse Path Filtering (http://lartc.org/howto/lartc.kernel.html), then the reply to your packets sent to the IP of the eth3 interface will be sent back through eth3 regardless of the contents of your routing table. So run:

echo 2 > /proc/sys/net/ipv4/conf/eth3/rp_filter

and then try the ping again.

However, you shouldn't have three default gateways with the same metric - maybe you should redesign the solution and define your goals clearer.

3
  • Still doesn't work after this, so I guess this does mean a problem with the network rather than the server.
    – Tim
    Oct 12, 2011 at 15:06
  • @Tim run a tcpdump on the eth3 interface: tcpdump -vvv -i eth3 host IP_OF_PINGING_HOST and icmp -nn and try to ping again.
    – ciupinet
    Oct 12, 2011 at 22:18
  • rp_filter has no effect on which interface a packet is received on -- The sender controls that, not the receiver. It also has no effect on which packet an interface is sent out on, the routing table (and policy, if any) decides that. This setting only affects a case that can never happen with this configuration, since rp_filter only affects a packet received on an interface a reply could not be sent on -- he has a default route out every interface, so there will never be a packet received on an interface a reply could not be sent on. (Default route means any packet could be sent that way.) Oct 13, 2011 at 10:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .