I have 2 machines on a 100 Gb/s LAN which talk to each other over TCP.
When I do a tracert or a ping from the client to the server, even a large ping, I get an answer in less then 1 ms.
However, when I'm sniffing the TCP packets on the client side, I see that 40ms pass between the time it sends the first message of the 3-way handshake, to the time it receives the answer from the server.
The server is dedicated to this client and handles nothing else.

Is this reasonable? I'd expect it to be much much faster.

  • How exactly is TCP session started? Are we talking about ping someplace.com vs. http to someplace.com or something else? – dbasnett Jul 29 '10 at 16:06
  • programatically opening a socket, with pooling. – Meidan Alon Aug 3 '10 at 13:42

Ping is typically handled internally by the TCP stack, but the handshake needs to be handled by the application. The OS has to

1) wake up the application, 
2) wait for it to retrieve the packet, 
3) do any pre-processing required when starting a new connection then 
4) tell the OS to continue with the handshake. 

Part 3 above can take a significant amount of time (this is why a lot of servers pre-fork). What kind of network traffic are you seeing the long handshake RTTs for? Does the RTT improve after the handshake?



What do you mean when you say large ping ? Large packet ? In any case, ping uses ICMP packets, which are usually small and with high priority. So, one case is to check your configurations.

Another thing is to try a non-icmp ping, like tcpping. It uses TCP syn packets, so it will be about the same.

  • large ping == large packet, I just wanted to see there're no routing or network problems. tcpping is a good idea, thanks. – Meidan Alon Jul 29 '10 at 8:52
  • 1
    -1: There is no such thing as 'priority' across a LAN. For packets to be prioritixed, they must be stored queued and forwarded - there is nothing on a LAN which does this - otherwise its a routed and not local network – symcbean Jul 29 '10 at 11:42
  • I found no mention of ICMP in the link you posted. ICMP is normally low priority by policy, not high. – dbasnett Jul 29 '10 at 16:04
  • Ok, i can agree with that. The link was just to mention the possibility of having QOS in link layer. – Nikolaidis Fotis Jul 29 '10 at 16:25

Setting TCP_NODELAY and combining a couple of messages to a single one did the trick, messages were just too small.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.