As sysadmin1138 says, you do need read access to the LDAP database. You can achive this by adding a special ldap user with read access to every attributes (except userPassword).
dn: cn=admin,dc=example,dc=com
objectClass: simpleSecurityObject
objectClass: organizationalRole
cn: admin
description: LDAP administrator
userPassword:: <some sha1 hash>
And then in the access control file (assuming OpenLDAP here):
access to *
by dn.regex="cn=admin,dc=example,dc=com" read
by * read
You could also give read access to anonymous users (only restricting userPassword). Then you want need a special admin user you can just drop libnss_ldap.secret and pam_ldap.secret. This works equally well, and the uid and gecos fields of your user database is rearly all that secret anyway. This is what I usually do. You might want to set size limits and restrict access to the mail attribute to authenticated users:
sizelimit 100
timelimit 60
access to attrs=userPassword
by anonymous auth
by * none
access to attrs=mail
by self read
by users read
by * none
access to *
by * read
Hope that helps!