4

We are getting 7% reset messages for our TCP layer messages. We have two sets of logs.

The first of Wireshark logs shows a good request and response Here the client was replied with sync acknowledgement such as [syn, ack] seq=0 ack=1
[syn, ack] seq=0 ack=1

The second set of log show that the server replied with SEQ=1 directly and ACK had a large ACK number to which the client which posting sent a reset message.

[ACK] seq=1 ack= large ack number

What server setting should we change on our Linux server to change the ACK messages so these resets can stop.

Improve this question

1 Answer 1

Reset to default
8

What you are seeing is normal behavior if a previous TCP connection did not get shut down properly.

If the port numbers match a previous connection which was completely closed on the client but remains established on the server, the sequence you see will happen.

The client will send a SYN, because it does not recall the old connection. The server will see the SYN arriving on a connection in established state, which is unexpected. But since it is a valid connection, the server cannot respond with a RST. Instead it will send an ACK corresponding to the current sequence numbers known to the server.

The client will receiver an ACK for a connection that is not yet established. First packet in each direction is required to have the SYN bit set. So the client knows the ACK is for a previous connection and will respond with a RST. The RST will clear the old connection from the server.

After the SYN, ACK, and RST packets have been send, the client can retransmit the SYN packet. This time the handshake will work.

There are multiple possible reasons for the old connection ending up in the broken state in the first place. Here are a few possibilities:

  • The network connection was down for a couple of minutes when the old connection was closed.
  • The client machine was rebooted.
  • One of the IP addresses was reassigned to a different machine while a connection was established.
  • You have a misbehaving middlebox somewhere between client and server (that could be a NAT or a firewall).
Improve this answer
4
  • Sorry for reply to old topic. How could NAT or firewall cause this? We also have a load balancer in the middle.
    – brian beuning
    Mar 4, 2019 at 16:46
  • 1
    @brianbeuning A NAT could cause this by forgetting the connection. Since a NAT has a limited amount of memory and limited port numbers, it sometimes has to drop connections. If the NAT then assigns the same port number to a new client, these symptoms can happen. A firewall could cause a similar effect by silently dropping some of the packets involved in shutting down a connection which could leave one endpoint not knowing the connection had been shut down.
    – kasperd
    Mar 4, 2019 at 16:57
  • FYI, in dealing with the same issue, the SYN-less ACK's large ack number doesn't seem to always correspond to the current sequence numbers known to the server. Instead, it seems to be a number that is extremely large. I've verified this with packet captures where I've got all the packets in/out of the server before the client reboots, and then the reconnection attempt by the client.
    – aggieNick02
    Sep 6, 2019 at 13:52
  • Also an interesting tidbit - this is what is supposed to happen to allow the client to reuse the ephemeral outbound port it chooses. The current Windows PE doesn't do this, which can lead to failures to connect. See stackoverflow.com/questions/57778683/…
    – aggieNick02
    Sep 6, 2019 at 13:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.