0

I'm investigating an issue where the application (Java based) didn't receive the whole message which was split on two TCP segments. I have a trace that proves that both segments was sent to the server.

From my investigation, I didn't find any dropped packets on the NICs but I noticed the following on netstat -s:

16 packets pruned from receive queue because of socket buffer overrun
845 packets collapsed in receive queue due to low socket buffer

I assume that the lost TCP segment could be one of these 16 pruned packets.

The question(s) here is the following:

Does it make sense to try to tune tcp_rmem? Should I expect to a well-tuned server/network pruned packets to be 0?

1 Answer 1

1

It is very unlikely that the problem needs to be solved at the system level instead of the application level. If something is lost in TCP it gets resent by the peer - this is how TCP is designed.

It is more likely that there are wrong assumption about how TCP works in the Java application. For example it is common error to treat TCP as a message based protocol instead of a byte stream and to assume that a single read will receive all the data which were just sent. Since this is not guaranteed this will result in the Java application sometimes reading only part of the message, even though the rest would have been eventually available for the application to read.

2
  • Problem seems to not be on the Java application since it's handling billion of messages per day and this happens once per day. I agree with you on how TCP works (When a packet is lost on TCP level, it gets resent by the peer) but this is not the case for the pruned packets. These are packets received by the OS and pruned/dropped by the kernel due to socket buffer overrun. These packets are lost and won't be resent. Oct 17, 2019 at 15:15
  • @PavlosMaragkos: An ACK gets only sent by the OS if the data could successfully be placed in the socket buffer of the application. If no ACK gets received by the peer the data gets resent. And it is possible that the application can almost ever read all the data it needs at once but that in a few cases it will fail so. Thus just because the application works fine most of the time is not an argument that the problem is not in the application. In fact, I've had to do with lots of such rare bugs and race conditions myself so I know that these happen. Oct 17, 2019 at 15:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .