SSH over NAT error :ssh_exchange_identification: read: Connection reset by peer

Question

My server is connecting through a router(NAT) to internet, I've set port forwarding already.

SSH work fine when connect in LAN IP, but not working when connect through WAN IP.

What I have tried

• restart service/computer
• change port (both on router and ssh port)
• reset ssh keys(those is /etc/ssh)
• checked /etc/hosts.allow and /etc/hosts.deny
• add sshd: ALL to hosts.allow
• check iptables -L (nothing in it)
• ProxyCommand nc %h %p from the answer here

server-side log (note: xxx.xxx.xxx.xxx is my public IP)

debug1: sshd version OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA
debug1: private host key: #0 type 1 RSA
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type DSA
debug1: private host key: #1 type 2 DSA
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type ECDSA
debug1: private host key: #2 type 3 ECDSA
debug1: private host key: #3 type 4 ED25519
debug1: rexec_argv[0]='/usr/sbin/sshd'
debug1: rexec_argv[1]='-d'
Set /proc/self/oom_score_adj from 0 to -1000
debug1: Bind to port 222 on 0.0.0.0.
Server listening on 0.0.0.0 port 222.
debug1: Bind to port 222 on ::.
Server listening on :: port 222.
debug1: Server will not fork when running in debugging mode.
debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8
debug1: inetd sockets after dupping: 3, 3
Connection from xxx.xxx.xxx.xxx port 58644 on 192.168.0.101 port 222

client-side log

ssh -vvv -C -A -X -p 2222 [email protected]
OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g  1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "xxx.xxx.xxx.xxx" port 2222
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 2222.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /home/yan/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/yan/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/yan/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/yan/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/yan/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/yan/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/yan/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/yan/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
ssh_exchange_identification: read: Connection reset by peer

Any advice would be appreciate. Thanks!

Answer 1

ssh_exchange_identification: read: Connection reset by peer

After lot of struggle, I fixed the ssh connection refused by simply running the following command.

sudo dhclient

Answer 2

Thank you for providing the logs. So we can see, that sshd on the server sees the connection come in, but taht's it.

My first guess is about your NAT router; maybe it doesn't do it's job well. Your description isn't clear in one respect: if you connecdt from the server to the internet side of your NAT router and this doesn't work, you might try to connect from the internet.

You might try other services, like a web server or anything else you know about, then adjusting your NAT router to make it accessable, too. Does it work or not?

Very exact information you can gather using a sniffing tool like tcpdump or wireshark. Ideally you use it on both sides of the connection; then you will see how packets are NATed.

Let's say your ssh client is on IP 1.1.1.1 and your server's internet connection sits on 3.3.3.3. So typing ssh 3.3.3.3 ENTER should send a tcp syn packet from 1.1.1.1 and random port, say 55555 to 3.3.3.3. You should see this packet on it's network interface using your sniffe4r. This packet travels through the internet to your NAT router on 3.3.3.3, where it gets NATed to a LAN IP, say 192.168.1.11. The sniffer at your server should see a tcp syn packet from 1.1.1.1 to 192.168.1.11 port 22.

When sshd accepts the connection, you should see the answer packet on your local sniffer: a tcp ack/syn packet from 192.168.1.11 port 22 to 1.1.1.1 port 55555. It travels through your NAT router and should get modified to "from 3.3.3.3 port 22 to 1.1.1.1 port 55555". You usually can't see the outgoing packet, because you can't sniff at this point. But if all went correct up to here, it will travel to your client side and there the sniffer can catch it. At this point your client will see the connection as established and answer with a packet from 1.1.1.1 port 55555 to 3.3.3.3 port 22 with a tcp ack packet to confirm the connection establishment. This packet will travle and get NATed and ... your server will see a the two-way handshake complete, too.

But as we see "Connection reset by peer", most probably your problem will show up before this point.

Just a hint for tcpdumping: you might see a neven ending stream of packets if you ssh to your client machine (might be a root server on the internet) and start tcpdum there - it will tcpdump the packets transporting your tcpdump output, which gets tcpdumped again and so on. To get this managed easily, you can change the port of your ssh server (or at least the NAT configuration on your router) to another port, say 2222.

Then your ssh command will be: ssh 3.3.3.3 -p 2222, and you can filter your tcpdump like

tcpdump port 2222

or, if you want to see all but your working ssh connection to the client:

tcpdump not port 22

Have fun tcpdumping and watching internet traffic and learning a lot doing so!

TomTomTom