I'm reading about TCP data flow, Delayed ACK and Nagle's Algorithm.

So far I understand that:

  1. The Delayed ACK implementation on TCP creates a delay on the acknowledgement of segments received to give the opportunity for the application to write some data along with the acknowledgement, thus avoiding sending an empty ACK packet and contributing to network congestion.
  2. The Nagle's Algorithm implementation states that you can't send a small TCP segment while another small segment is still not acknowledged. This avoids the traffic being loaded with several tinygrams.

On some interactive applications, like Rlogin for instance, Nagle's Algorithm and Delayed ACKs can "conflict":

Rlogin sends the keyboard input to the server as we type them and some keys (like F1) generates more than one byte (F1 = Escape + left bracket + M). Those bytes can be sent in different segments if they are delivered to TCP one by one.

The server doesn't reply with an echo until it has the whole sequence, so all the ACKs would be delayed (expecting some data from the application). The client on the other hand, would wait for the first byte acknowledgement before sending the next one (respecting the Nagle's Algorithm). This combination ends up resulting in a "laggy" Rlogin.

The tcpdump of the F1 and F2 key being sent on a Rlogin is represented below:

    type Fl key
1   0.0                 slip.1023 > vangogh. login: P 1:2(1) ack 2
2   0.250520 (0.2505)   vangogh.login > slip.1023: P 2:4(2) ack 2
3   0.251709 (0.0012)   slip.1023 > vangogh.login: P 2:4(2) ack 4
4   0.490344 (0.2386)   vangogh.login > slip.1023: P 4:6(2) ack 4
5   0.588694 (0.0984)   slip.1023 > vangogh.login: . ack 6
    type F2 key
6   2.836830 (2.2481)   slip.1023 > vangogh.login: P 4:5(1) ack 6
7   3.132388 (0.2956)   vangogh.login > slip.1023: P 6:8(2) ack 5
8   3.133573 (0.0012)   slip.1023 > vangogh.login: P 5:7(2) ack 8
9   3.370346 (0.2368)   vangogh.login > slip.1023: P 8:10(2) ack 7
10  3.388692 (0.0183)   slip.1023 > vangogh.login: . ack 10

Now the doubt: Even though the page I read states that the server doesn't reply with an echo before it has the whole key sequence, the packets captured through tcpdump shows that the keys are being echoed on their respective ACKs (the first reply is 2 bytes long because the echo from ESC is two characters - caret + left bracket).

If data is being sent from the application to TCP (the echo response) why are the ACKs being delayed? According to what was stated, about the server waiting the full sequence before echoing it, wasn't the ACKs supposed to contain no echo up to the last ACK, that would contain the whole sequence echo?

Reference: http://people.na.infn.it/~garufi/didattica/CorsoAcq/Trasp/Lezione9/tcpip_ill/tcp_int.htm


You say rlogin "server doesn't reply with an echo until it has the whole sequence" such as the usual ^[OP for F1 key. But that's just an unwarranted assumption. And a wrong one. And your tcpdump experiment shows this is not the case; it shows exactly a lagging rlogin implementation without any such 'optimization'.

Indeed, a normal, expected behavior for server is to immediately echo any input. If client for some reason decides to send ^[ alone, it won't experience a weird lag after that ((leaving aside the TCP completely).

What should be a clean solution to the laggy-rlogin class of problems, is that each side of conversation (both client and server) only send() when they honestly believe that at this very moment some human is waiting for the result to be displayed. Under such constraint, it's a major bug in the client to send ^[, when the software already knows that ^[OP is the full intended sequence and a human user is interested in the complete result, not just the result of ^[ (does a human decide to either send OP or maybe OQ in response, or what?).

So, the counter-intuitive advice for software developers, whatever is their business with TCP and either they develop client side or server side: remember that send() is not immediate, it will possibly delay your next transfer, so use it more carefully (a result of Nagle adding latency with delayed ACK adding even more latency).

  • I'm picking your answer since it kind of complements the one I accepted on StackOverflow. I believe this book's chapter is a bit misleading, both on taking the assumption that the echo response doesn't happen until you have the whole sequence (as you stated) and the assumption that the ~200ms delay is due to the ACKs being delayed. Comparing both tcpdump outputs it looks like the ~200ms delay could be caused by the network link transfer rate (as stated in the SO answer) instead. – IanC Oct 6 '16 at 11:15
  • In the end, the example only shows the issues of using Nagle's algorithm with programs that exchanges tinygrams frequently, without necessarily being related with delayed ACKs (even though when present those could worsen the problem). I'd just suggest an edit: removing this line - "with delayed ACK adding even more latency" - since apparently the ACKs aren't delayed after all. – IanC Oct 6 '16 at 11:17

AFAIK the Nagle's Algorithm is set on very low level of the socket configuration. For example C/C++ implementations of the socket communication are using quite low level API to switch it off.

The idea of Nagle's is pretty good in typical usage of the TCP/IP.

For example we had been using some messaging libraries for the soft real-time system and we had explicitly switched it off on the socket level. Exactly for that reason. We do not want to wait. With Nagle'S socket waits until the complete buffer is full - regardless of the type of the message. So ACKs are delayed as all of them are small and buffer will slowly be full.

But practical hints are following: 1. Switch off Nagle’s Algorithm or 2. If you cannot, then send bigger messages, so Nagle’s buffer is often full.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.