10

I have a Slackware linux box where I cannot start any service that listens on one particular port on localhost. By using strace I found out that the error happens on the bind() call, and the error is EADDRINUSE (Address already in use):

bind(3, {sa_family=AF_INET, sin_port=htons(874), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EADDRINUSE (Address already in use)

This happens with any process I try to start listening on that port, so it is not related to the process itself. The above strace output comes from the command strace -ff nc -l -p 874 -s 127.0.0.1.

So, this suggests there is a process already listening on localhost port 874. However, I can't seem to find it. The following commands all return nothing:

netstat -aplunt | grep :874
netstat -na | grep :874
lsof -i :874
lsof -i tcp | grep 874
fuser 874/tcp
socklist | grep 874
iptables -t filter -S | grep 874
iptables -t nat -S | grep 874
iptables -t mangle -S | grep 874
conntrack -L | grep 874

If I try to listen on 0.0.0.0:874 it fails with the same error. Listening on one of the IP addresses configured on a nic works OK, and listening to 127.0.0.2:874 also works OK. Listening on a different port works fine, also on 127.0.0.1 or 0.0.0.0.

So, now I am curious. How can I find out why the network stack returns EADDRINUSE here? What other things could I look at, or what other commands can I run to get more information?

Additional info:

  • Kernel 4.1.31.
  • Selinux is not used here.
  • Trying to connect to 127.0.0.1 with telnet returns "Connection refused"
  • I'm running the commands as root
| improve this question | | | | |
  • 1
    Is the port mentioned anywhere in iptables -S output? – hertitu Oct 24 '16 at 21:40
  • 1
    Can you please also print output of strace -ff nc -l 874 also, The one you used is trying to make connections with 874 as source port. Thanks! – Anirudh Malhotra Oct 25 '16 at 7:12
  • 2
    AFAIK Linux requires root privileges when listening to a port < 1000. Maybe that's the problem here. – Koraktor Oct 25 '16 at 7:38
  • 2
    Is this host an NFS client? It may be using source port 874 for an NFS mount. Anyway I would try netstat -na | grep 874 in case your current netstat flags are too restrictive. – Tom Shaw Oct 25 '16 at 23:52
  • 1
    @TomShaw You just made my day! It was NFS all right. I'm not sure how this exactly came about, but after unmounting all NFS mounts and restarting the RPC and NFS services the problem is gone. With tcpdump I also saw some traffic from port 874 to port 111 (why didn't I think of that before) which confirms it. Can you post this as an answer please? – roelvanmeer Oct 26 '16 at 6:19
4

If your host is an NFS client, it may be using source port 874 for an NFS mount. I suspect that because the connection does not originate from userspace it may not be visible to the tools you've used so far.

Consider one of the following:

  • Adjust the sysctls sunrpc.min_resvport and sunrpc.max_resvport (default 665 and 1023) to change the range of source ports that the NFS client uses
  • Use a listening port outside of this range
  • Use the noresvport option on the NFS mount to use the non-privileged range (may have security implications)
| improve this answer | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.