Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a CentOS 5.5 server that is using 1 of the 2 on-board network ports (eth0) and 2 PCI NICs (eth2 & eth3) - each has it's own IP on a different subnet with different gateway on a different VLAN. The IPs on eth0 & eth2 are working fine, but the IP on eth3 is not responding to requests including ICMP, though it does show as active on the switch and if I ping the IP locally on the server it does work.

I am guessing this some kind of IP routing issue with Linux, maybe it's trying to respond to eth3 requests on eth2 or eth0 instead of eth3?

Hopefully someone has some suggestions for me please?

eth0: x.x.236.2 netmask 255.255.252.0 gateway x.x.236.1
eth2: x.x.232.2 netmask 255.255.252.0 gateway x.x.232.1
eth3: x.x.208.2 netmask 255.255.248.0 gateway x.x.208.1

Output from "ip route":

x.x.236.0/22 dev eth0  proto kernel  scope link  src x.x.236.2
x.x.232.0/22 dev eth2  proto kernel  scope link  src x.x.232.2
x.x.208.0/21 dev eth3  proto kernel  scope link  src x.x.208.2
169.254.0.0/16 dev eth3  scope link
default via x.x.232.1 dev eth2
default via x.x.236.1 dev eth0
default via x.x.208.1 dev eth3
share|improve this question
    
disconnect the other two interfaces. Can you reach eth3? –  Zoredache Oct 12 '11 at 15:32
add comment

2 Answers

up vote 3 down vote accepted

Linux sends outbound packets on the interface that is closest to the destination. It doesn't consider a packet a 'response' to another packet for routing purposes, nor would it make sense for it to do so.

You should not typically have three gateways, except for redundancy. If a router cannot handle a packet from any of the machine's IP addresses to any host that you don't have a more-specific route to, it should not be listed as a gateway.

There are no such thing as "eth3 requests". There are just packets, with source and destination IP addresses.

Three listed defaults means three machines that can handle any packet to a non-local host.

share|improve this answer
    
So if it's only 1 of the IPs that isn't working this suggests it's a problem with the router? –  Tim Oct 12 '11 at 13:52
1  
It suggests that the router isn't correctly configured to handle packets with that source IP address. If it's supposed to, then the problem is with the router. If it's not supposed to, then the problem is that you have a special requirement but a standard configuration. The standard assumption is that a router can handle a packet with any source address so long as its destination is one the router is supposed to handle. –  David Schwartz Oct 12 '11 at 15:24
1  
Thank you the problem was found to be a faulty switch port it has now been changed to a different port and the server has been re-configured to use only 1 default gateway for all 3 NICs and everything is working perfectly. –  Tim Oct 13 '11 at 11:02
    
Wow. Sometimes when you hear hoofbeats behind you, it really is zebras. –  David Schwartz Oct 13 '11 at 11:08
add comment

If you disable Reverse Path Filtering (http://lartc.org/howto/lartc.kernel.html), then the reply to your packets sent to the IP of the eth3 interface will be sent back through eth3 regardless of the contents of your routing table. So run:

echo 2 > /proc/sys/net/ipv4/conf/eth3/rp_filter

and then try the ping again.

However, you shouldn't have three default gateways with the same metric - maybe you should redesign the solution and define your goals clearer.

share|improve this answer
    
Still doesn't work after this, so I guess this does mean a problem with the network rather than the server. –  Tim Oct 12 '11 at 15:06
    
@Tim run a tcpdump on the eth3 interface: tcpdump -vvv -i eth3 host IP_OF_PINGING_HOST and icmp -nn and try to ping again. –  ciupinet Oct 12 '11 at 22:18
    
rp_filter has no effect on which interface a packet is received on -- The sender controls that, not the receiver. It also has no effect on which packet an interface is sent out on, the routing table (and policy, if any) decides that. This setting only affects a case that can never happen with this configuration, since rp_filter only affects a packet received on an interface a reply could not be sent on -- he has a default route out every interface, so there will never be a packet received on an interface a reply could not be sent on. (Default route means any packet could be sent that way.) –  David Schwartz Oct 13 '11 at 10:56
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.