I could not ssh into one of my servers this morning after successfully doing so most recently yesterday.

ssh hostname
Connection closed by hostname port 22

I rebooted system and it did not fix the issue.

I dug out a keyboard and a display and could not find any issues or messages in /var/log/messages /var/log/warn or journalctl.

I stopped sshd (systemctl stop sshd), ran sshd from the command line (sshd -ddd) and there are no error messages, and no connection. sshd does unexpectedly terminate with an exit value 255. Note that when running as a system daemon it is not exiting, just not reporting anything.

Another symptom is that ssh does not prompt for a password for either a non-existing user or an existing user. sshd will however log that the non-existing user is invalid:

Invalid user bob from [host] port 40718

While investigating this issue, I found that usermod fails.

usermod -a -G nogroup nobody
usermod: PAM: Authentication failure

Thinking this is a symptom of my sshd failure I started to investigate usermod failing and found the deep pit of despair that is PAM. This question Switch on PAM debugging to Syslog documents the zero ability to debug anything within PAM without recompling from source.


I think this error is a result of a incomplete patch or patch dependency that got installed yesterday. Resolved by installing patches.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.