Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Background:
My company is switching to an MPLS provider. The MPLS provider is providing both internet and MPLS. This is done with one ethernet drop by using two dot1q sub interfaces, one for the private MPLS portion and one for the internet.

The old setup was a bunch of site-to-site IPSEC VPN tunnels going over the internet and static routes. The new setup is using eigrp.

Problem:
The performance of the new MPLS network seems to be slower. There is a web application used internally that has pages that take 10 seconds longer to load then they used to. The odd thing is, that when I stop using the MPLS, and set up a site-to-site vpn tunnel over the new internet connection (same physical ethernet ports), that is also faster. The provider said that the tunnel ends up taking pretty much the same routes as the MPLS would.

What we have tired and found:
Ping latency is stable and decent. Traceroutes look normal. Everything still works, but seems to be slower. The provider and us don't see packet loss or errors on the interface stats. There were some clocking issues but they fixed it.

Where to go from Here:
I know networking probably requires a lot of detail about the setup, so really I am just looking for ideas and things to look at and consider. The next step I am going to take with the provider is to get cisco support on the case, and both my company and our provider our going to open or routers to them. But in the meantime, I am open to any ideas to consider looking into as to why this might be.

share|improve this question
    
Oh ya, and CPU load looks fine. –  Kyle Brandt Sep 11 '09 at 12:33

3 Answers 3

I would look for an MTU issue.

With an MPLS tunnel the packets must be slightly larger than the payload -- 4 bytes for the MPLS label, for each encapsulation (ie if you're going to put MPLS tunnels into an MPLS tunnel, the outside tunnel will be 8 bytes larger than the payload). Here's a discussion of this.

Hopefully you're not filtering ICMP messages -- if you are you'll be dropping the "use a smaller MTU" packets the network will be sending to your hosts if indeed you are sending too-large packets over the wire. This discusses issues related to MTU...

One thing you can try on your end without involving your ISP is you can shrink your MTU on your border device -- that way it negotiates a proper MTU with your inside network for traffic going over the MPLS link.

share|improve this answer
    
elifulkerson.com/projects/mturoute.php is a free util to help Test MTU on windows. –  Jeremy Rossi Sep 11 '09 at 13:34

I would take a tcpdump/wireshark capture at both side of your provider MPLS network to look if packet are identical. You may look for missing packet, for fragmentation (mtu problem), ...

share|improve this answer
    
Ya, Good point maybe I need to set this up. This would have been the first thing I did if I had some Linux boxes. The particular box that is slow is a old Windows 2000 box that is too fragile for me to put wireshark on, and I am not there to be able to set up a mirror port on the switch, but maybe I will see if I can get the done remotely. –  Kyle Brandt Sep 11 '09 at 12:49
    
Can you expand on how to identify fragmentation or mtu problems? –  Kyle Brandt Sep 11 '09 at 12:49
    
if MTU is too small, packet will be framgmented, so you will see 1 packet at the sending side but 2 (or more) on the other side. –  radius Sep 11 '09 at 14:05
2  
MTU can be easily troubleshoot with a ping. on windows do ping -f -l X <IP>. find the X value where it stop working. If it's less than 1472 you have a MTU problem –  radius Sep 11 '09 at 14:10
    
packets may be fragmented. They may also simply be dropped with an icmp sent to the sender of MTU too large. And if you're using MPLS, you may not even get that because the MPLS packet is what gets dropped and the thing that drops it won't see inside the packet to know who to send the error to. –  chris Sep 11 '09 at 15:18
up vote 0 down vote accepted

Eventually trying a different interface fixed the problem, even cisco support couldn't say exactly what was going on...

share|improve this answer
    
Sounds like a physical issue with the port, but I must say I have a hard time articulating a reasoning behind that. –  Vatine Oct 2 '10 at 9:30

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.