/usr/bin/sshd isn't linked against PAM on one of my systems. What is wrong and how can I fix it?

Question

I'm using AD as my user account server with ldap.

Most of the servers run with UsePam yes except this one,

it has lack of pam support on sshd.

root@linserv9:~# ldd /usr/sbin/sshd 
    linux-vdso.so.1 =>  (0x00007fff621fe000)
    libutil.so.1 => /lib/libutil.so.1 (0x00007fd759d0b000)
    libz.so.1 => /usr/lib/libz.so.1 (0x00007fd759af4000)
    libnsl.so.1 => /lib/libnsl.so.1 (0x00007fd7598db000)
    libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 (0x00007fd75955b000)
    libcrypt.so.1 => /lib/libcrypt.so.1 (0x00007fd759323000)
    libc.so.6 => /lib/libc.so.6 (0x00007fd758fc1000)
    libdl.so.2 => /lib/libdl.so.2 (0x00007fd758dbd000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fd759f0e000)

I have this packages installed

root@linserv9:~# dpkg -l|grep -E 'pam|ssh'
ii  denyhosts                             2.6-2.1                      an utility to help sys admins thwart ssh hac
ii  libpam-modules                        0.99.7.1-5ubuntu6.1          Pluggable Authentication Modules for PAM
ii  libpam-runtime                        0.99.7.1-5ubuntu6.1          Runtime support for the PAM library
ii  libpam-ssh                            1.91.0-9.2                   enable SSO behavior for ssh and pam
ii  libpam0g                              0.99.7.1-5ubuntu6.1          Pluggable Authentication Modules library
ii  libpam0g-dev                          0.99.7.1-5ubuntu6.1          Development files for PAM
ii  openssh-blacklist                     0.1-1ubuntu0.8.04.1          list of blacklisted OpenSSH RSA and DSA keys
ii  openssh-client                        1:4.7p1-8ubuntu1.2           secure shell client, an rlogin/rsh/rcp repla
ii  openssh-server                        1:4.7p1-8ubuntu1.2           secure shell server, an rshd replacement
ii  quest-openssh                         5.2p1_q13-1                  Secure shell
root@linserv9:~# 

What I'm doing wrong?

thanks.

Edit:

root@linserv9:~# cat  /etc/pam.d/sshd 
# PAM configuration for the Secure Shell service

# Read environment variables from /etc/environment and
# /etc/security/pam_env.conf.
auth       required     pam_env.so # [1]
# In Debian 4.0 (etch), locale-related environment variables were moved to
# /etc/default/locale, so read that as well.
auth       required     pam_env.so envfile=/etc/default/locale

# Standard Un*x authentication.
@include common-auth

# Disallow non-root logins when /etc/nologin exists.
account    required     pam_nologin.so

# Uncomment and edit /etc/security/access.conf if you need to set complex
# access limits that are hard to express in sshd_config.
# account  required     pam_access.so

# Standard Un*x authorization.
@include common-account

# Standard Un*x session setup and teardown.
@include common-session

# Print the message of the day upon successful login.
session    optional     pam_motd.so # [1]

# Print the status of the user's mailbox upon successful login.
session    optional     pam_mail.so standard noenv # [1]

# Set up user limits from /etc/security/limits.conf.
session    required     pam_limits.so

# Set up SELinux capabilities (need modified pam)
# session  required     pam_selinux.so multiple

# Standard Un*x password updating.
@include common-password

Edit2: UsePAM yes fails

With this configuration ssh fails to start :

root@linserv9:/home/admmarc# cat /etc/ssh/sshd_config |grep -vE "^[ \t]*$|^#"
Port 22
Protocol 2
ListenAddress 0.0.0.0
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile      .ssh/authorized_keys
ChallengeResponseAuthentication yes
UsePAM yes
Subsystem       sftp    /usr/lib/sftp-server
root@linserv9:/home/admmarc# 

The error it gives is as follows

root@linserv9:/home/admmarc# /etc/init.d/ssh start
 * Starting OpenBSD Secure Shell server sshd
/etc/ssh/sshd_config: line 75: Bad configuration option: UsePAM
/etc/ssh/sshd_config: terminating, 1 bad configuration options
   ...fail!
root@linserv9:/home/admmarc#

Answer 1

It looks like your /usr/sbin/sshd binary has been overwritten.

This could mean you have had a security breach, or maybe someone just compiled a version locally and the Ubuntu version was overwritten.

The 1:4.7p1-8ubuntu1.2 (x86) version of the openssh-server definitely is definitely linked against libpam. It is conceivable that the pam support was left out of the 64bit version, but that seems unlikely.

I'd try reinstalling openssh-server:

# First back up the current binary
$ sudo cp /usr/sbin/sshd /root/sshd.bak

# reinstall the Ubuntu version
$ sudo apt-get install --reinstall openssh-server

# compare the two versions
$ sudo sha1sum /usr/sbin/sshd /root/sshd.bak

# In my case, they match:
# 8a3ccd5242380674bc45b887286faa3abb51acdb  /usr/sbin/sshd
# 8a3ccd5242380674bc45b887286faa3abb51acdb  /root/sshd.bak

If yours don't match each other (not mine), there is definitely something fishy going on and you really need to figure out where that version of sshd came from.

If they do match, then I am probably wrong and it is a bug in that version of openssh-server for 64-bit Ubuntu.

Answer 2

Does /etc/pam.d/sshd exist? It's part of openssh-server.

Answer 3

Obviously you have already set "UsePAM" to yes in /etc/ssh/sshd_config file? right?!?

Answer 4

If your system was really compromised, a system reinstall might be in order!