1

Can somebody give me ideas about following situation (Linux 2.6.18-348.4.1.el5 ):

  1. At some point tcpdump shows [FIN, ACK] packet sent from server port to local client
  2. strace log shows no socket activity performed on that socket handle for that port near that moment (strace log properly shows the rest of communication for that client)
  3. Firewall and SELinux are stopped
  4. Problem is 100% reproducible with complex conditions which require the Server to perform network request for Kerberos authentication of another client connection.

What are potential causes which may lead tcpdump to show packet which missing in strace? Does it look more like server problem, TCP settings problem or some firewall service problem?

Improve this question
8
  • 1
    Are you experiencing a problem, or is this hypothetical?
    – Halfgaar
    Commented Jul 10, 2014 at 9:53
  • Yes, this FIN packet aborts connection, which becomes a problem. I need to determine where to dig - application, TCP settings, or some problem ssytem service (like firewall)
    – noonex
    Commented Jul 10, 2014 at 9:59
  • Can you try connecting the server and client directly (or with one switch, or whatever)? Do FIN packages normally show up on strace? Could there perhaps be an 3rd (malicious) client sending spoofed FIN packages?
    – Halfgaar
    Commented Jul 10, 2014 at 10:11
  • No, FIN packets don't normally appear in an strace. You might see a call to shutdown() or close() on the socket, and the kernel would end up closing the connection, but you can't see individual packets in the TCP stream in strace.
    – Flup
    Commented Jul 10, 2014 at 10:20
  • 1
    I am guessing multiple processes/threads have the file descriptor open, and one of them is calling shutdown on the socket. You may not be seeing this in strace simply because you are not looking at the right thread. With lsof it should be possible to find out, which processes has file descriptors for that socket.
    – kasperd
    Commented Jul 10, 2014 at 10:54

2 Answers 2

Reset to default
0

Something triggers the closing of the connection on the server side. Most likely it is the application on the remote server that shuts down the connection. Possible reasons are idle disconnection feature on the server, software bugs etc.

If the remote server is behind NAT, it could also be caused by expiration of the NAT entry in the device doing the NAT.

You should run tcpdump on the server and see if the FIN packet is originated from that server. If it is, then the likely culprit is the application. If not, then it is the from firewall.

Improve this answer
1
  • The problem is reproduced locally with client connecting from the same host.
    – noonex
    Commented Jul 10, 2014 at 10:50
0

kasperd was right about strace not showing all threads of the process (somehow I was sure that attaching to process will monitor all threads). So auditd confirmed it is my application closing the socket (because some misterious "signal 33" happens and recv() doesn't handle EINTR properly. I will ask another question about this).

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .