How to debug slow network pings

Question

Our hosted servers are experiencing periods of very slow network access. During these periods pinging from any of our servers to the default gateway results in ping times of up to 500ms (instead of the normal 2-3ms). There are no packets lost.

The ISP is claiming their network is OK and the problem must be with the hosting service. The hosting service guys claim they have nothing at level 3 and everything according to them is OK.

I'm think our servers are connected to a switch which is connected the default gateway over a shared ATM line. Traceroute doesn't show any routers on the path.

Before changing my service providers, is there anything I could check that might explain what is causing this issue?

Answer 1

Routers don't prioritize handling ICMP to themselves. Pinging a router and getting high latency means nothing (well, not what you want anyway). What happens if you ping a unix system one router-hop away? Or since we are dealing with 500ms if that is the latency then that will show up clearly if you ping anyone on you continent.

Answer 2

You might also want to try mtr (mtr-tiny is a nice package I have on every server).

It will give you details about all the hops that are between you and the target. Give it a try.

Answer 3

Are pings equally slow using hostname and IP address? This could be an issue of a slow DNS server.

Answer 4

PingPlotter will give you graphs of traceroute data over time so you can see exactly where and at what time you are getting problems.

Answer 5

Normally, I'd avoid pinging the default gateway to measure delay. Answering ICMP is about the least-prioritised task that routers do, so all that 500 ms delay means is that the router's busy (may not even be busy routing packets, if lots of people are pinging it, because they think that's a good way of gauging the network).

If at all possible, find a host on the other side of the default gateway and ping that. It gives you a much better idea if there's anything wrong.

One of the reasons that sort of thing (answering ICMP EHO requests and other non-essential traffic handling) is de-prioritised is to make it harder to cause a denial of service, simply by smurfing the router.

Answer 6

You could try arping as an alternative to the oft prioritised ICMP echo.

Answer 7

Pings are accurate only as far as ICMP is left to its defaults on their end, even then its really only an indication that something is alive and connected to the network.

ICMP is (almost) never left to its defaults and is sometimes ignored completely. If you want a better test, set up a simple echoreply service on your server then measure the round trip time for a packet filling {insert frame size here}.