33

On a VM I am initializing I am able to log in as one non-root user (admin) but not another (tbbscraper) over SSH with public key authentication. The only error message I can find in any log file is

Sep 18 17:21:04 [REDACTED] sshd[18942]: fatal: Access denied for user tbbscraper by PAM account configuration [preauth]

On the client side, the syndrome is

$ ssh -v -i [REDACTED] tbbscraper@[REDACTED]
...
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: [REDACTED]
debug1: Authentications that can continue: publickey
debug1: Trying private key: [REDACTED]
debug1: read PEM private key done: type RSA
Connection closed by [REDACTED]

Changing 'tbbscraper' to 'admin' allows a successful login: debug1: Authentication succeeded (publickey). appears instead of the "Connection closed" message.

This doesn't seem to be a permissions problem...

# for x in admin tbbscraper
> do ls -adl /home/$x /home/$x/.ssh /home/$x/.ssh/authorized_keys
> done
drwxr-xr-x 3 admin admin 4096 Sep 18 17:19 /home/admin
drwx------ 2 admin admin 4096 Sep 18 16:53 /home/admin/.ssh
-rw------- 1 admin admin  398 Sep 18 17:19 /home/admin/.ssh/authorized_keys
drwxr-xr-x 3 tbbscraper tbbscraper 4096 Sep 18 17:18 /home/tbbscraper
drwx------ 2 tbbscraper tbbscraper 4096 Sep 18 17:18 /home/tbbscraper/.ssh
-rw------- 1 tbbscraper tbbscraper  398 Sep 18 17:18 /home/tbbscraper/.ssh/authorized_keys

# cmp /home/{admin,tbbscraper}/.ssh/authorized_keys ; echo $?
0

... nor a PAM-level access control problem ...

# egrep -v '^(#|$)' /etc/security/*.conf
#

... so none of the existing answers to similar questions would seem to apply. The only other piece of evidence I've got is:

root@[REDACTED] # su - admin
admin@[REDACTED] $

but

root@[REDACTED] # su - tbbscraper
su: Authentication failure
(Ignored)
tbbscraper@[REDACTED] $

which suggests some larger-scale PAM issue, but I can't find anything obviously wrong with the stuff in /etc/pam.d. Any ideas?

The VM is an EC2 instance, OS is Debian 7.1 (Amazon's off-the-shelf AMI).

2
  • /etc/pam.d/sshd please
    – GioMac
    Commented Sep 18, 2013 at 17:47
  • @GioMac Never mind, I found the problem.
    – zwol
    Commented Sep 18, 2013 at 17:59

8 Answers 8

40

After all that, it turns out to have been a one-character typo in /etc/shadow. Spot the difference:

admin:!:15891:0:99999:7:::
tbbscraper:!::15966:0:99999:7:::

That's right, there are two colons after the exclamation point on the tbbscraper line. That shoves all the fields over one and makes PAM think that the account expired on January 8, 1970.

2
  • 10
    Thanks for posting. This was useful for me: I'd manually created a user entry in /etc/passwd and forgotten to add a corresponding /etc/shadow entry.
    – spazm
    Commented Jul 7, 2014 at 19:07
  • 7
    @spazm Thanks for commenting. This was useful for me: I'd manually copied users from another machine and forgotten to copy the /etc/shadow entry of the one user without a password.
    – Jayen
    Commented Jan 1, 2015 at 5:49
11

Thank you for posting your question. I was getting the same error, but my problem wasn't related to the shadow file. I found my fix and wanted to post an answer as well for anyone else Googling this error. This serverfault question comes up first.

Try checking the /etc/security/access.conf !

We're using Active Directory for authentication, but I needed to login in as a local, non-AD user (jenkins). My boss had originally setup the box with these line in the /etc/security/access.conf:

+:root:ALL
-:ALL:ALL

I changed it to the following and logins now work; I didn't even need to restart any services.

+:jenkins:ALL
+:root:ALL
-:ALL:ALL
5

In my case I was renaming local CentOS 6 users, and forgot to rename them in /etc/shadow (who are password-less key-authenticated, didn't pop up in my mind), so the records for the new usernames were just absent in /etc/shadow. In /var/log/secure it was giving me unix_chkpwd error and Access denied by PAM:

    unix_chkpwd[12345]: could not obtain user info (user2)
    sshd[12354]: fatal: Access denied for user user2 by PAM account configuration
1
  • 1
    usermod(8) is your friend next time ;-) Commented May 8, 2018 at 6:56
5

Had the same error message. Shutted down the sshd and restarted it in debug mode

    /usr/sbin/sshd -ddd

this indicated the reason:

    debug3: User autossh not allowed because account is locked
            ...
    input_userauth_request: invalid user <username> [preauth]

Checked account:

    passwd -S <username>

which showed that the account was locked (flag "L") Un-locked the account by setting a new password:

    passwd <username>

Done.

3

I got the same problem this morning but the server authenticates users against Active Directory. Turns out the user's domain password had expired.

1
  • 3
    Same phenomenon, different source of user account information :-) It's possible that I should have filed a bug against ssh and/or PAM two years ago, asking for clearer logging of why a login attempt was denied; there is a security argument for not telling the person who made the attempt why it failed, but that wouldn't apply to system logs.
    – zwol
    Commented Nov 11, 2015 at 16:34
2

I had the same issue and none of suggested options worked. But I found in one of forums (https://ubuntuforums.org/showthread.php?t=1960510) a "workaround" which worked perfectly.

Edit /etc/ssh/sshd_config and set

UsePAM no

While it's probably not the real solution, because something is definitely wrong with my machine (yesterday it was working fine!), this one at least works.

1
  • That worked for me, inherited a few machines where some had that issue. Commented Mar 1, 2022 at 9:06
1

The accepted answer helped me find my error. The accepted answer show and extra colon between the 'encrypted password' field and the 'date of last password change' field. See man shadow

user1:!:15891:0:99999:7:::
user2:!::15966:0:99999:7:::

Where the OP's problem shifted the fields by one, mine were in the correct place, but there was an extra field at the end.

user1:<removed-to-shorten-the-line>:19744:0:99999:7:::
user2:<removed-to-shorten-the-line>:19745:0:99999:7::::

Once corrected on the target machine, the ssh session worked as expected.

The usermod utility was not as immediately helpful since the shadow file was in a system image that is pushed to hundreds of nodes. (I could have used the usermod in a chroot environment, but sometimes it is quicker to make the change by hand in the image.)

However, on a node running the "broken" image the error can be detected using the passwd utility.

the problem showed up with this output:

passwd -S user2
user2 P

Once /etc/shadow was corrected:

passwd -S user2
user2 P 01/01/2023 0 99999 7 -1
0

In my case it was junk hitting ''/etc/tcb/USER/shadow'' after ext4 rootfs corruption in "interesting" conditions; it looked pretty texty so wasn't spotted during initial examination (can't reinstall the node right now but will have to).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .