Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Can anyone with heavy background in both Linux and Networking help me out? I've scoured the Internet looking for a clear answer for this and I've even compared a box I have access to with the same setup and I can't get my box working.

The basic situation is this:

===========================

A RHEL 6 Linux box has two Ethernet interfaces:

eth0 = 10.10.5.10/24
eth1 = 10.10.6.10/24

[root@box ~]# netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
10.10.5.0      0.0.0.0         255.255.255.0   U         0 0          0 eth0
10.10.6.0      0.0.0.0         255.255.255.0   U         0 0          0 eth1
0.0.0.0        10.10.5.254     0.0.0.0         UG        0 0          0 eth0

ip forwarding has been enabled and is active.

The problem:

Clients on the same subnet as eth1 or the 10.10.6.x network can ping and telnet to resources running on eth1 on this box.

However, clients on any other subnet cannot. So from my workstation I can ping and telnet to resources on the box's eth0 all day long, but nothing on eth1... What gives?

Comparing this setup to another box of a similar nature in my environment shows no discernible differences.

Here are some files:

# cat ifcfg-eth0
DEVICE="eth0"
BOOTPROTO="none"
HWADDR="Omitted"
IPADDR="10.10.5.10"
NETMASK="255.255.255.0"
NETWORK="10.10.5.0"
ONBOOT="yes"
TYPE="Ethernet"
USERCTL="no"

# cat ifcfg-eth1
DEVICE="eth1"
BOOTPROTO="none"
HWADDR="omitted"
IPADDR="10.10.6.10"
NETMASK="255.255.255.0"
NETWORK="10.10.6.0"
ONBOOT="yes"
TYPE="Ethernet"
USERCTL="no"

# cat route-eth0 
default via 10.10.5.254 dev eth0

# cat /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=box
GATEWAY=10.10.5.254
NOZEROCONF=yes

Any assistance you can provide will be appreciated!

Thanks.

Adding more info, but with brevity included:

# ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    inet 10.10.5.10/24 brd 10.10.5.255 scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    inet 10.10.6.10/24 brd 10.10.6.255 scope global eth1
share|improve this question
    
is your eth0 interface up? can you reach anything on the 10.10.5.0/24 network? Can you reach your default gateway 10.10.5.254? if you can't for either these, then you won't be able to get to anything outside of your 10.10.6.0/24 network, as your server doesn't know how to get outside of its local network. –  emynd Jun 6 '13 at 15:20
    
Yes, anything on the 5 network is reachable from the same subnet or a different subnet. –  beaconfield Jun 6 '13 at 15:22
    
i am specifically talking about from your server. Can your server reach its default gateway? –  emynd Jun 6 '13 at 15:24
    
After running a tcpdump I can see packets hitting the eth1 interface, so it looks like the packets are having an issue returning? Shouldn't they use the default gateway? And if they are, should they be using a different gateway somehow? –  beaconfield Jun 6 '13 at 15:24
    
@emynd Yes it can reach its default gateway. It has connectivity to its network on the switch and the larger network within the infrastructure. No problems there. –  beaconfield Jun 6 '13 at 15:24

3 Answers 3

with the setup as you currently show:

  • you have 2 sides :

    • the eth1 side (10.10.6.10 interface, on 10.10.6.0/24 lan)
    • the eth0 side (10.10.5.10 interface, on 10.10.5.0/24 lan)
    • you defined that machine's default gateway on eth0 side (default gw ip: 10.10.5.254)
  • therefore

    • on eth1 side:
      • [requisite] any machines in 10.10.6.0/24 should be able to communicate with 10.10.6.10 (and with each other as well) (I assume no firewalling issues)
      • [requisite] in addition: any machine in 10.10.6.0/24 THAT HAS defined their default gateway as "10.10.6.10" will be able to send packets to other machines, going through your machine (10.10.6.10/24->eth1[machine]eth0-> other lans). Actually you don't need to have it as default gateway, but for ANY lan on the eth0 side, they need to associate that LAN with the 10.10.6.10 gateway.
    • on eth0 side:
      • [requisite] any machine on 10.10.5.0/24 should see 10.10.5.10 (eth0) and can communicate with it (and with each other).
      • [requisite] any machine on another LAN should be able to communicate with eth0, via (with maybe additionnal hops) the machine which has 10.10.5.254 ip (ex: if that machine have 2 subnets, 10.10.5.254/24 on the same side as eth0, and 10.x.y.z/24 on another interface, machine in 10.x.y.0/24 can send packets to 10.10.5.10 by having a route: 10.10.5.0/24 with gateway 10.x.y.z).
      • if so, then they just need to have an additionnal route to reach 10.10.6.0/24 : 10.10.6.0/24 via gateway 10.x.y.z

it can get complicated to put in words...

what would help: tell us the list of networks ( ) and each network's getways.

right now I guessed you try to have:

 [everything other machines (even internet, via the exterior internet gateway)]
                  |
                  |(links via switch(s) for 10.x.y.0/24 network, for example.
                  |   If you have such an additionnal LAN.
                  |   Maybe it's even "the Internet", and therefore not in 10.x anything,  
                  |   but this is just an example.)
                  |
                  |      
      (10.x.y.z?) |  [---------------]
   (or Internet?)ethZ[some router/box]ethW(10.10.5.254)
                     [---------------] |
                                       |
                                       |(links via switch(s) for 10.10.5.0/24 network)
                                       |
                             [-------] |
             (10.10.6.10)eth1[yourbox]eth0(10.10.5.10)
                          |  [-------]   
                          |
                          |(links via switch(s) for 10.10.5.0/24 network)
                          |
                          |  [---------------------------]
                         ethx[any machine in 10.10.6.0/24]
                             [---------------------------]

General way to debug your setup:

  • imagine you ARE the IP packet: you have a source IP, a destination IP.
  • Now, on the interface you sit on at the source, see what that machine's routes tells you which next hop is. And go there(but first ensure that machine's setup allows you to, for example on the eth0/eth1 machine, it needs ipforward to go from one interface to the other). And proceed, one hop at a time, toward your goal (AND back to your source : sometimes the destination machine's route/defaultgw are broken and therefore the reply packet is not sent via the right path)
    • ex1 (very simple): 10.10.6.10 trying to send to 10.10.5.10: step1 you are on eth1 (10.10.6.10). That machine has a route saying "10.10.5.0/24 is directly here, on eth0". Ipforwarding allows you to hop from eth1 to eth0. Proceed to eth0. step2: you are now (same IP packet) on eth0. It turns out this is your destination: GOOD. step 2b: reply packet: inverse source/destination : you are now a packet (10.10.5.10 -> 10.10.6.10=): thank god, a local route tells you the next hop is on eth1. step4: no eth1 : you are at destination. GOOD.
    • ex2 (a bit longer ex:) you are a packet from "any_machine", 10.10.6.12 (on the eth1 side). you try to reach as destinatino google (8.8.8.8, for example). So you are "10.10.6.12 -> 8.8.8.8" and at first you sit on the 10.10.6.12 interface (say, eth3). A local route on "any_machine" tells you (as it was the prerequisite, above): "default gw=10.10.6.10", so you hop to "yourbox", and arrive there on the eth1 side. step2: you are now on "yourbox/eth1" : now local routes tell you "you need to go to default gw: 10.10.5.254, via eth0" (and ipforward on "yourbox" allows you to hop from eth1 to eth0). Now you arrive on eth0, and following the default gw route, you arrive later on "somerouter" on ethW(10.10.5.254). etc... (and don't forget to check the way-back route)
share|improve this answer
    
thank you for your response. I'm still looking it over, but: if I ping -I eth1 10.10.6.254 it works. If I ping -I eth1 10.10.5.254 it does not work. The reverse is true of pinging from eth0. It can ping its default gateway, but cannot ping eth1's default gateway of 10.10.6.254 on the switch. –  beaconfield Jun 6 '13 at 17:42
    
eth0 - 10.10.5.0/24 def. gateway = 10.10.5.254 –  beaconfield Jun 6 '13 at 17:43
    
eth1 - 10.10.6.0/24 def. gateway = 10.10.6.254 –  beaconfield Jun 6 '13 at 17:44
    
2 default gateways for your box ?? so if you try to reach, say, google, you send it both to eth1 to 10.10.6.254 AND to eth0 to 10.10.5.254 ??? –  Olivier Dulac Jun 6 '13 at 17:46
1  
let us continue this discussion in chat –  beaconfield Jun 6 '13 at 17:51

Your default gateway is at 10.10.5.254, however the eth1 interface is configured as 10.10.6.10/24. What this effectively means is that the 10.10.6.x interface cannot hit the default gateway as they are on different subnets.

More general network information is needed to properly assess your options, however your two options from what has been posted so far are to:

  • Confirm that the two networks are in fact /24 subnets, if so, set the proper gateway for eth1 in that interface's config file
  • If the interfaces are part of a larger supernet, configure the two interfaces to exist on a /16 subnet to use the same gateway.
share|improve this answer
    
thanks for your reply - the subnets are in fact /24 netmasks. So what are you saying? Define a line in the ifcfg-eth1 that says: GATEWAY=10.10.6.254 or are you saying I need to add some sort of static route? –  beaconfield Jun 6 '13 at 17:18
    
Exactly that, as long as 10.10.6.254 is indeed your gateway for that /24. It's preferred to do that for the other interface/network too. Further routing rules may need to be added to define the default routes for the system as a whole. –  NcA Jun 6 '13 at 17:21
    
@NcA : you probably confuse his box (having 2 interfaces, with the default gw on eth0) with a regular box with only 1 interface (therefore needing routes and/or a default gateway to reach other networks) : on his box, if ip_forward is 1, packets from eth1 can reach other lan's by going via eth0 (and its defautl gw)... IF you add a default gw on eth1 as well, packets by default will go through BOTH (and you better have a sane network behind, with each default gw knowing to avoid loops, to avoid hairy routing problems and superflous communications) –  Olivier Dulac Jun 6 '13 at 18:15

Have you enabled IP forwarding?

sysctl net.ipv4.ip_forward

If it says 0 you have not and your box is not routing packets.

Enable it by

sysctl -w net.ipv4.ip_forward=1

And make it permanent by adding the following to /etc/sysctl.conf:

net.ipv4.ip_forward = 1
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.