Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I've been ssh'ing to this box without problem for a year using public-key authentication.

No changes have been made to the client or server's ssh config. Now, ssh connections never complete or fail, they just hang. The following is the debug output when trying to connect:

OpenSSH_6.0p1 Debian-3ubuntu1, OpenSSL 1.0.1c 10 May 2012
debug1: Reading configuration data /home/jivan/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to <host> [<ip>] port <port>.
debug1: Connection established.
debug1: identity file .ssh/id_rsa.pub type 1
debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
debug1: identity file .ssh/id_rsa.pub-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.8p1 Debian-7ubuntu1
debug1: match: OpenSSH_5.8p1 Debian-7ubuntu1 pat OpenSSH_5*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.0p1 Debian-3ubuntu1
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA 81:63:05:0c:39:6b:77:d7:a2:25:93:30:80:f1:ba:0e
debug1: Host '[<host>]:<port>' is known and matches the ECDSA host key.
debug1: Found key in /home/jivan/.ssh/known_hosts:39
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: .ssh/id_rsa.pub
debug1: Server accepts key: pkalg ssh-rsa blen 279

Server log:

Feb 24 12:02:38 balance sshd[5734]: debug1: Forked child 6745.
Feb 24 12:02:38 balance sshd[6745]: Set /proc/self/oom_score_adj to 0
Feb 24 12:02:38 balance sshd[6745]: debug1: rexec start in 5 out 5 newsock 5 pip
e 7 sock 8
Feb 24 12:02:38 balance sshd[6745]: debug1: inetd sockets after dupping: 3, 3
Feb 24 12:02:38 balance sshd[6745]: Connection from 207.204.228.55 port 57967
Feb 24 12:02:38 balance sshd[6745]: debug1: Client protocol version 2.0; client 
software version OpenSSH_6.0p1 Debian-3ubuntu1
Feb 24 12:02:38 balance sshd[6745]: debug1: match: OpenSSH_6.0p1 Debian-3ubuntu1
 pat OpenSSH*
Feb 24 12:02:38 balance sshd[6745]: debug1: Enabling compatibility mode for prot
ocol 2.0
Feb 24 12:02:38 balance sshd[6745]: debug1: Local version string SSH-2.0-OpenSSH_5.8p1 Debian-7ubuntu1
Feb 24 12:02:39 balance sshd[6745]: debug1: PAM: initializing for "jivan"
Feb 24 12:02:39 balance sshd[6745]: debug1: PAM: setting PAM_RHOST to "207.204.228.55"
Feb 24 12:02:39 balance sshd[6745]: debug1: PAM: setting PAM_TTY to "ssh"
Feb 24 12:02:40 balance sshd[6745]: debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
Feb 24 12:02:40 balance sshd[6745]: debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
Feb 24 12:02:40 balance sshd[6745]: debug1: temporarily_use_uid: 1000/1000 (e=0/0)
Feb 24 12:02:40 balance sshd[6745]: debug1: trying public key file /home/jivan/.ssh/authorized_keys
Feb 24 12:02:40 balance sshd[6745]: debug1: fd 4 clearing O_NONBLOCK
Feb 24 12:02:40 balance sshd[6745]: debug1: matching key found: file /home/jivan/.ssh/authorized_keys, line 1
Feb 24 12:02:40 balance sshd[6745]: Found matching RSA key: 5f:8c:76:ad:48:6a:f5:73:03:6a:36:13:c0:ea:3b:45
Feb 24 12:02:40 balance sshd[6745]: debug1: restore_uid: 0/0

Any ideas what the problem could be?

Update: The problem stopped as mysteriously as it started. Still interested in any insight as to what might have caused this.

share|improve this question
    
It would be more helpful to see the log of the server. To get it you should change the LogLevel directive in /etc/ssh/sshd_config on the SSH server, then restart it. Then try reconnecting again and check the server logs. –  mauro.stettler Feb 24 '13 at 11:59
    
Did you give it a few minutes to finish? It could have been your DNS server was down and it was waiting for the reverse DNS to time out –  kormoc Feb 25 '13 at 1:19
    
I think I saw a similar problem caused by a broken openssl.cnf on the server-side but this is just a wild guess in your case. –  scai Feb 25 '13 at 7:26

3 Answers 3

up vote 2 down vote accepted

This is usually caused by an issue with reverse DNS; just had this happen to one of my legacy servers the other week. You were right to invoke ssh -d, but the reverse DNS problems won't really become apparent unless you run it with ssh -ddd instead so that you're seeing the output of debug3.

While reverse DNS issues don't always cause things to back up this badly, if enough DNS lookups related to the connection attempt are failing (i.e. all DNS servers are completely unavailable, or enough of the related zones/reverse zones are unavailable), under some configurations it will be enough to completely lock out ssh access to the system. It doesn't matter how high you set your client timeout if the server gets impatient and drops the connection.

share|improve this answer

While the situation above is most likely DNS related, I kept experiencing stalls right after

debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

it was not consistent either. It went away by doing:

KexAlgorithms diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1

I suspect the real origin is limited MTU of IPv6 tunnel, but I have yet to verify this. Google finds this question.

share|improve this answer

I've had this problem too, but here it was the client's turn to talk. In this case, it appeared to be due to the presence of a gpg-agent (running in ssh-agent-support mode); killing it or ssh-add -D'ing it allowed resumption of ssh connections.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.