I have a Linux based router with four interfaces (each with its own private subnet).
When I directly connect a device directly (i.e. no switch, just a patch cable) to one interface and another device directly to another, as below, then the router works perfectly.
DEVICE1
192.168.8.11 ------- 192.168.8.254
ROUTER
10.58.129.254 ------- DEVICE2
10.58.129.1
When I connect the router with our switches in between, as below, then router doesn't work.
DEVICE1
192.168.8.11 ----------- switch1
|
switch2
|
switch3
|
192.168.8.254
ROUTER
10.58.129.254 -------- switch3
|
DEVICE2
10.58.129.1
All the switches are Layer 3, Switch1 (Dell PowerConnect 3548P) has a fibre connection to Switch2 (Dell PowerConnect 6224F) which is our core switch that handles routing between most VLANs. This is connected via fibre to Switch3 (Dell PowerConnect 6224).
Routing on the core switch isn't enabled either of the two VLANs (192.168.8.11 or 10.58.129.254). The reason for this is because our core switch doesn't support policy based routing, hence the reason behind this Linux box to perform routing on these VLANs.
With the router connected via the switches, from Device1, I can ping the interface 192.168.8.254 on the Linux router, but can't ping the other interface (10.58.129.254).
Switch2 configuration/diagnostics
switch2#show ip route
Route Codes: R - RIP Derived, O - OSPF Derived, C - Connected, S - Static
B - BGP Derived, IA - OSPF Inter Area
E1 - OSPF External Type 1, E2 - OSPF External Type 2
N1 - OSPF NSSA External Type 1, N2 - OSPF NSSA External Type 2
S 0.0.0.0/0 [50/0] via 10.58.3.16, vlan 3
C 10.58.3.0/24 [0/0] directly connected, vlan 3
C 10.58.4.0/24 [0/0] directly connected, vlan 4
C 10.58.5.0/24 [0/0] directly connected, vlan 5
C 10.58.9.0/24 [0/0] directly connected, vlan 9
C 10.58.10.0/24 [0/0] directly connected, vlan 10
C 10.58.11.0/24 [0/0] directly connected, vlan 11
C 10.58.12.0/24 [0/0] directly connected, vlan 12
S 10.58.64.0/24 [40/0] via 10.58.3.17, vlan 3
S 10.58.128.0/24 [40/0] via 10.58.3.254, vlan 3
S 10.58.129.0/24 [1/0] via 10.58.3.254, vlan 3
S 192.168.8.0/24 [1/0] via 10.58.3.254, vlan 3
switch2#ping 10.58.129.254
Pinging 10.58.129.254 with 64 bytes of data:
----10.58.129.254 PING Statistics----
4 packets transmitted,0 packets received,100% packet loss
round-trip (ms) min/avg/max = 0/NaN/0
switch2#ping 192.168.8.254
Pinging 192.168.8.254 with 64 bytes of data:
----192.168.8.254 PING Statistics----
4 packets transmitted,0 packets received,100% packet loss
round-trip (ms) min/avg/max = 0/NaN/0
Router diagnostics
router# traceroute -d 192.168.8.11
traceroute to 192.168.8.11 (192.168.8.11), 30 hops max, 60 byte packets
1 192.168.8.11 (192.168.8.11) 0.237 ms 0.222 ms 0.211 ms
router# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.58.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.58.128.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
10.58.129.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
192.168.8.0 0.0.0.0 255.255.255.0 U 0 0 0 eth4
router# ping 192.168.8.11
PING 192.168.8.11 (192.168.8.11) 56(84) bytes of data.
64 bytes from 192.168.8.11: icmp_seq=1 ttl=128 time=2.23 ms
64 bytes from 192.168.8.11: icmp_seq=2 ttl=128 time=0.237 ms
Device1 diagnostics
(device1)c:\>route print
===========================================================================
Interface List
0x1 ........................... MS TCP Loopback interface
0x2 ...bc 30 5b d8 41 c3 ...... Broadcom NetXtreme 57xx Gigabit Controller - Pac
ket Scheduler Miniport
===========================================================================
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.8.254 192.168.8.11 20
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
192.168.8.0 255.255.255.0 192.168.8.11 192.168.8.11 20
192.168.8.11 255.255.255.255 127.0.0.1 127.0.0.1 20
192.168.8.255 255.255.255.255 192.168.8.11 192.168.8.11 20
224.0.0.0 240.0.0.0 192.168.8.11 192.168.8.11 20
255.255.255.255 255.255.255.255 192.168.8.11 192.168.8.11 1
Default Gateway: 192.168.8.254
===========================================================================
Persistent Routes:
None
(device1)c:\>tracert -d 10.58.129.254
Tracing route to 10.58.129.254 over a maximum of 30 hops
1 * * * Request timed out.
2 * * * Request timed out.
3 * * * Request timed out.
4 * * * Request timed out.
(etc. until 30 hops).
So from device1, running ping 10.58.129.254
, and with tcpdump
running on the 192.168.8.254 interface of the Linux router, I can see ICMP echo requests and replies
router# tcpdump -i eth4
17:08:08.326221 IP 192.168.8.11 > 10.58.129.254: ICMP echo request, id 512, seq 63746, length 40
17:08:08.326240 IP 10.58.129.254 > 192.168.8.11: ICMP echo reply, id 512, seq 63746, length 40
But the reply never returns to device1.
Does anybody know what the problem might be? tcpdump on eth2,3 & 4 also reveals the following output (I've not seen it on eth0, which is the one VLAN of the above that is routed by the core switch):
19:49:16.246286 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id
8000.a4:ba:db:69:74:91.8014, length 43
19:49:18.257007 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id
8000.a4:ba:db:69:74:91.8014, length 43
I understand this is spanning tree, but I don't know if this is a bad thing or not. Does this offer any clues? For info, the hardware address in the above STP message, is that of switch3.