I'm hitting a very strange problem - when I try to connect to one of our servers the client hangs with SYN_SENT if something listens on the port (e.g. Apache on port 80, sshd on port 22 or SMTP on port 25) but if I try to connect to a port on which nothing listens then I immediately get a "CONNECTION refused" error. Connecting to other applications (e.g. rsyncd on some arbitrary port) succeeds.
I ran tcpdump on the server and see that the SYN packets arrive to it but it only sends a response if nothing listens on that port.
e.g.: on the server I run:
# tcpdump -nn port 81 06:49:34.641080 IP 10.x.y.z.49829 > server.81: S 3966400723:3966400723(0) win 12320 06:49:34.641118 IP server.81 > x.y.z.49829: R 0:0(0) ack 3966400724 win 0
But if I listen on this port, e.g. with
nc -4lvvv 81 &
Then the output of tcpdump is:
06:44:31.063614 IP x.y.z.45954 > server.81: S 3493682313:3493682313(0) win 12320
(and repeats until I stop it)
The server is CentOS 5, the client is Ubuntu 11.04, the connection is done between two LAN's over per-user TCP OpenVPN.
Connection to other servers on that network do not have a problem. Connecting from the other servers on the same network to that server works fine. Connections from other clients in our office over openvpn is also not a problem.
What am I missing?