Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have 4 servers running Centos 5. All of them have two ethernet network cards.

I have configured 192.168.1.x IP addresses on their eth0 card. They are all connected to the same switch using their eth0 card and they are all working.

I have configured 10.72.11.x IP addresses on their eth1 card.They are all connected to the same switch - a different one from the switch used with eth0 card - using their eth1 card and they are NOT all working.

Their configuration files is like:

DEVICE=eth1
BOOTPROTO=static
IPADDR=10.72.11.236
BROADCAST=10.72.11.191
NETMASK=255.255.255.192
NETWORK=10.72.11.128
HWADDR=84:2B:2B:55:4B:98
IPV6INIT=yes
IPV6_AUTOCONF=yes
ONBOOT=yes

The interfase is starting and configured as I need. [root@sql1 network-scripts]# ifconfig
eth0 Link encap:Ethernet HWaddr 84:2B:2B:55:4B:97
inet addr:192.168.1.105 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::862b:2bff:fe55:4b97/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2981 errors:0 dropped:0 overruns:0 frame:0
TX packets:319 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:386809 (377.7 KiB) TX bytes:66134 (64.5 KiB)
Interrupt:36 Memory:da000000-da012800
eth1 Link encap:Ethernet HWaddr 84:2B:2B:55:4B:98
inet addr:10.72.11.236 Bcast:10.72.11.191 Mask:255.255.255.192
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:48 Memory:dc000000-dc012800

I also added a route-eth1 file that looks like:
10.0.0.0/8 via 10.72.11.254

Routing looks fine to me:

[root@sql1 network-scripts]# netstat -rn  
Kernel IP routing table  
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface  
10.72.11.192    0.0.0.0         255.255.255.192 U         0 0          0 eth1  
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0  
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth1  
10.0.0.0        10.72.11.254    255.0.0.0       UG        0 0          0 eth1  
0.0.0.0         192.168.1.1     0.0.0.0         UG        0 0          0 eth0  

But I cannot ping one server from the other.

[root@sql1 network-scripts]# ping 10.72.11.235
PING 10.72.11.235 (10.72.11.235) 56(84) bytes of data.
From 10.72.11.236 icmp_seq=1 Destination Host Unreachable
From 10.72.11.236 icmp_seq=2 Destination Host Unreachable
From 10.72.11.236 icmp_seq=3 Destination Host Unreachable
From 10.72.11.236 icmp_seq=4 Destination Host Unreachable
From 10.72.11.236 icmp_seq=5 Destination Host Unreachable
From 10.72.11.236 icmp_seq=6 Destination Host Unreachable
^C
--- 10.72.11.235 ping statistics ---
7 packets transmitted, 0 received, +6 errors, 100% packet loss, time 6033ms , pipe 3

What am I doing wrong?

share|improve this question
add comment

5 Answers 5

up vote 1 down vote accepted

It should be noted that your NICs aren't properly 'working'. Your ifconfig output doesn't show the 'RUNNING' required for there to be a properly working link. note the line UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 from your eth0 interface. There's no 'RUNNING' evident in the eth1 output. In my experience, this indicates a low level issue. eg: Bad/disconnected cabling.

Is is possible the NIC you have plugged your cable into is NOT eth1? Servers can get some weird naming going on. Try checking the output of /var/log/messages, to see if it reports an interface getting (dis)connected. Likewise, see if you get any diagnostic info there when you connect/disconnect, or during boot. Check and see if this is occurring on all 5 boxes. You may also want to try the graphical configuration tool, just in case you've got a typo in the configuration file. (system-config-network)

share|improve this answer
    
You were right cables were wrong. –  Sebas Feb 9 '11 at 0:53
add comment

Think a bit about your subnet setup:

IPADDR=10.72.11.236
BROADCAST=10.72.11.191
NETMASK=255.255.255.192
NETWORK=10.72.11.128

In base 2:

236 is 1110 1100
191 is 1011 1111
192 is 1100 0000
128 is 1000 0000

Your IPs are outside your subnet - 10.72.11.236 is not in 10.72.11.128/26, 10.72.11.172 is. Neither is 17.72.11.235 but 10.72.11.171 is. Either change your IPs or change your broadcast to .255 and your network to .192. Alternatively (unless it's being dictated to you), run in the lower half of 10.72.11.0 (i.e. 10.72.11.0/26) or just use a /24 and save yourself the hassle.

share|improve this answer
    
It appears you got 2 separate segments going on your proposed single segment switching network. Unless you are running vlans on that switch, it may be confused depending on your configuration and in this specific configuration its going to be. Trying bumping your 10.72.11.X networks all to 24 bits. –  Jason B Shrout Feb 7 '11 at 23:38
add comment

Try using ipcalc from http://jodies.de/ipcalc, it is a big help when configuring networks and subnetworks.

# ipcalc 10.72.11.236/255.255.255.192
Address:   10.72.11.236         00001010.01001000.00001011.11 101100
Netmask:   255.255.255.192 = 26 11111111.11111111.11111111.11 000000
Wildcard:  0.0.0.63             00000000.00000000.00000000.00 111111
=>
Network:   10.72.11.192/26      00001010.01001000.00001011.11 000000
HostMin:   10.72.11.193         00001010.01001000.00001011.11 000001
HostMax:   10.72.11.254         00001010.01001000.00001011.11 111110
Broadcast: 10.72.11.255         00001010.01001000.00001011.11 111111
Hosts/Net: 62                    Class A, Private Internet

Richard

share|improve this answer
    
That is what I used. Thanks. –  Sebas Feb 8 '11 at 20:35
add comment

I've seen this problem before. Not 100% sure of the fix, but I remember there being something in sysctl to tell ARP to consider these different devices - or something along these lines.

There have been a few posts about in on serverfault before if I remember correctly.

share|improve this answer
add comment

Inbound traffic that goes to eth0 should come back via eth0 and vice versa. Try doing a TCPdump to check where traffic is going.

You also need policy based routing to do this : See Linux 2 network card routing depending on the interface used

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.