Sign up ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a Linux based router with four interfaces (each with its own private subnet).

When I directly connect a device directly (i.e. no switch, just a patch cable) to one interface and another device directly to another, as below, then the router works perfectly.

   DEVICE1 ------- 
            ------- DEVICE2

When I connect the router with our switches in between, as below, then router doesn't work.

   DEVICE1 ----------- switch1
             -------- switch3

All the switches are Layer 3, Switch1 (Dell PowerConnect 3548P) has a fibre connection to Switch2 (Dell PowerConnect 6224F) which is our core switch that handles routing between most VLANs. This is connected via fibre to Switch3 (Dell PowerConnect 6224).

Routing on the core switch isn't enabled either of the two VLANs ( or The reason for this is because our core switch doesn't support policy based routing, hence the reason behind this Linux box to perform routing on these VLANs.

With the router connected via the switches, from Device1, I can ping the interface on the Linux router, but can't ping the other interface (

Switch2 configuration/diagnostics

switch2#show ip route

Route Codes: R - RIP Derived, O - OSPF Derived, C - Connected, S - Static
       B - BGP Derived, IA - OSPF Inter Area
       E1 - OSPF External Type 1, E2 - OSPF External Type 2
       N1 - OSPF NSSA External Type 1, N2 - OSPF NSSA External Type 2

S [50/0] via,   vlan 3
C [0/0] directly connected,   vlan 3
C [0/0] directly connected,   vlan 4
C [0/0] directly connected,   vlan 5
C [0/0] directly connected,   vlan 9
C [0/0] directly connected,   vlan 10
C [0/0] directly connected,   vlan 11
C [0/0] directly connected,   vlan 12
S [40/0] via,   vlan 3
S [40/0] via,   vlan 3
S [1/0] via,   vlan 3 
S [1/0] via,   vlan 3

Pinging with 64 bytes of data:
---- PING Statistics----
4 packets transmitted,0 packets received,100% packet loss
round-trip (ms) min/avg/max = 0/NaN/0

Pinging with 64 bytes of data:
---- PING Statistics----
4 packets transmitted,0 packets received,100% packet loss
round-trip (ms) min/avg/max = 0/NaN/0

Router diagnostics

router# traceroute -d
traceroute to (, 30 hops max, 60 byte packets
 1 (  0.237 ms  0.222 ms  0.211 ms

router# route -n
Kernel IP routing table
Destination     Gateway      Genmask         Flags Metric Ref    Use Iface   U     0      0        0 eth0   U     0      0        0 eth3   U     0      0        0 eth2   U     0      0        0 eth4

router# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=128 time=2.23 ms
64 bytes from icmp_seq=2 ttl=128 time=0.237 ms

Device1 diagnostics

(device1)c:\>route print
Interface List
0x1 ........................... MS TCP Loopback interface
0x2 ...bc 30 5b d8 41 c3 ...... Broadcom NetXtreme 57xx Gigabit Controller - Pac
ket Scheduler Miniport
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
       20       1       20       20       20       20       1
Default Gateway:
Persistent Routes:

(device1)c:\>tracert -d
Tracing route to over a maximum of 30 hops
  1     *        *        *     Request timed out.
  2     *        *        *     Request timed out.
  3     *        *        *     Request timed out.
  4     *        *        *     Request timed out.
(etc. until 30 hops).

So from device1, running ping, and with tcpdump running on the interface of the Linux router, I can see ICMP echo requests and replies

router# tcpdump -i eth4
17:08:08.326221 IP > ICMP echo request, id 512, seq 63746, length 40
17:08:08.326240 IP > ICMP echo reply, id 512, seq 63746, length 40

But the reply never returns to device1.

Does anybody know what the problem might be? tcpdump on eth2,3 & 4 also reveals the following output (I've not seen it on eth0, which is the one VLAN of the above that is routed by the core switch):

19:49:16.246286 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id
8000.a4:ba:db:69:74:91.8014, length 43
19:49:18.257007 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id
8000.a4:ba:db:69:74:91.8014, length 43

I understand this is spanning tree, but I don't know if this is a bad thing or not. Does this offer any clues? For info, the hardware address in the above STP message, is that of switch3.

share|improve this question
When you run traceroute from either end, how far does the trace get? –  Rory Alsop May 9 '11 at 16:59
+1 for the traceroute. Also, care to add the routing table from DEVICE1, ROUTER and DEVICE2 ? Also your problem pinging the remote interface of a multi-homed gateway reminds me of a specific problem with some linux, I'll check it. –  Renik May 9 '11 at 17:08
Ditto on the traceroute, but also from the reverse direction as well. On the same vein of thought, what/where are your references for your current ping outcomes? Have you also tried different reference points across your network in order zero in? –  user48838 May 9 '11 at 17:20
Thanks for the comments. I left the office a few hours ago, and I can only access the router from here. (I've updated the question with what little I can add at the moment). When I return to work in the morning, I'll add the remainder of the requested info to the question. –  Bryan May 9 '11 at 18:37
I'm willing to bet that the icmp requests and replies are coming and leaving from different interfaces. Check with a tcpdump on each interface separately. –  JimB May 9 '11 at 18:43

2 Answers 2

up vote 3 down vote accepted

In your second topology, you appear to have a split subnet. spans multiple switches which you state are layer 3. In your output from switch2, you have a static route for this /24 pointing at a single interface:

S [1/0] via,   vlan 3

This means that traffic hitting switch2 destined for either or will be forwarded to the same next hop. At least one of those destinations

In order for this to work as you intend, you have a couple of options:

  1. Configure switch[123] as layer 2 switches, such that the is again a single broadcast domain.
  2. Configure separate networks on each of the links: {device1 -> switch1, switch1 -> switch2, etc}
share|improve this answer
+1: Thank Murali. I'm unable to test that now, as I've ended up using a different solution (as per my own answer), but I'm going to accept your answer anyway, as it does seem like that rouge static route would have been the cause of the problem. –  Bryan May 12 '11 at 9:31

Due to not being able to resolve this problem, a different approach is now being utilised which will vastly simplify the configuration. Instead of routing some VLANs by the core switch, and others by a Linux box, I'm now allowing the core switch to perform all routing between VLANs, with ACLs performing the segregation that two routers would have provided.

I'm going to move the Linux box to being the default gateway to the outside world for the entire network, with the help of iproute2, to perform source IP based routing, so the correct systems use the correct gateways.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.