I am currently trying to set up LDAP replication between to instances of 389 Directory Server (both running on Fedora 37), which I'll call $SUPPLIER and $CONSUMER in the following (serving at the domains supplier.mydomain.example and consumer.mydomain.example, respectively).
Both $SUPPLIER and $CONSUMER are configured identically and use Let's Encrypt certificates. I have successfully configured several clients of $SUPPLIER, which all work like a charm (all use ldaps:// to communicate with $SUPPLIER, and demand TLS certificates from it).
However, replicating from $SUPPLIER to $CONSUMER does not work. I receive the following error message:
[02/May/2023:14:28:07.389193770 +0200] - ERR - NSMMReplicationPlugin - bind_and_check_pwp - agmt="cn=Synchronize with $CONSUMER" (ldap3:636) - Replication bind with SIMPLE auth failed: LDAP error -1 (Can't contact LDAP server) (error:0A000086:SSL routines::certificate verify failed (unable to get issuer certificate)) [02/May/2023:14:28:10.408516918 +0200] - ERR - slapi_ldap_bind - Could not send bind request for id [cn=replication manager,cn=config] authentication mechanism [SIMPLE]: error -1 (Can't contact LDAP server), system error -5987 (Invalid function argument.), network error 0 (Unknown error, host "consumer.mydomain.example:636")
How do I resolve the “unable to get issuer certificate” error?
Most of the content I have already found about the problem assumes that self-signed certificates are used and that the CA needs to be manually installed; however I thought the entire joke of having a CA like Let's Encrypt issue a certificate was that the CA is already trusted by all operating systems / applications (e.g. like it is the case with browsers, where I can automatically connect to a Let's Encrypt-secured website).
Things that work
email@example.com$ dsconf -D "cn=replication manager,cn=config" ldaps://consumer.mydomain.example:636 monitor server dn: cn=monitor version: 389-Directory/2.3.3 ...
Any other valid
dsconf (or probably also
dsidm) commands also work, i.e. there is no error with the encrypted connection here.
firstname.lastname@example.org$ echo -n "" | openssl s_client -showcerts consumer.mydomain.example:636 CONNECTED(00000003) depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1 verify return:1 depth=1 C = US, O = Let's Encrypt, CN = R3 verify return:1 depth=0 CN = consumer.mydomain.example verify return:1 --- Certificate chain 0 s:CN = consumer.mydomain.example i:C = US, O = Let's Encrypt, CN = R3 a:PKEY: id-ecPublicKey, 256 (bit); sigalg: RSA-SHA256 v:NotBefore: Apr 10 18:33:38 2023 GMT; NotAfter: Jul 9 18:33:37 2023 GMT -----BEGIN CERTIFICATE----- [...] -----END CERTIFICATE----- 1 s:C = US, O = Let's Encrypt, CN = R3 i:C = US, O = Internet Security Research Group, CN = ISRG Root X1 a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256 v:NotBefore: Sep 4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT -----BEGIN CERTIFICATE----- [...] -----END CERTIFICATE----- 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1 i:O = Digital Signature Trust Co., CN = DST Root CA X3 a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256 v:NotBefore: Jan 20 19:14:03 2021 GMT; NotAfter: Sep 30 18:14:03 2024 GMT -----BEGIN CERTIFICATE----- [...] -----END CERTIFICATE----- --- Server certificate subject=CN = consumer.mydomain.example issuer=C = US, O = Let's Encrypt, CN = R3 --- No client certificate CA names sent Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512 Shared Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512 Peer signing digest: SHA256 Peer signature type: ECDSA Server Temp Key: X25519, 253 bits --- SSL handshake has read 4205 bytes and written 424 bytes Verification: OK --- New, TLSv1.3, Cipher is TLS_AES_128_GCM_SHA256 Server public key is 256 bit Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated Early data was not sent Verify return code: 0 (ok) --- DONE
email@example.com$ trust list [...] pkcs11:id=%79%B4%59%E6%7B%B6%E5%E4%01%73%80%08%88%C8%1A%58%F6%E9%9B%6E;type=cert type: certificate label: ISRG Root X1 trust: anchor category: authority [...]
Things that don't work
firstname.lastname@example.org$ echo -n | openssl s_client -connect consumer.mydomain.example:636 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/ldapserver.pem email@example.com$ openssl verify /tmp/ldapserver.pem CN = consumer.mydomain.example error 20 at 0 depth lookup: unable to get local issuer certificate error ldapserver.pem: verification failed
This seems to be the same error message as in the LDAP server's log, so maybe the two problems are related?
Update: Per dave_thompson_085's comment, this seems to be more of a Red Herring, as this command will apparently always fail, so it failing here is no surprise.
- Unable to establish replication with STARTTLS - 389-users - Fedora Mailing Lists: Problem seems to be very similar (and quite recent), not yet resolved.
- Old Let's Encrypt Root Certificate Expiration and OpenSSL 1.0.2: Might have been the problem if this was two years ago, however, I wouldn't expect it to cause any trouble on a modern OS in 2023.
- Unable to
openssl verifyletsencrypt certificate suggests to use the
-untrustedoption when using
openssl verify, however, I cannot pass this option to 389 DS, and I also don't really want to put
-untrustedflags anywhere near a production environment.
0A000086seems to be an error code emitted by OpenSSL (see e.g. this web search), so the problem is probably one in OpenSSL and not with 389 Directory Server directly?
openssl s_client >file; openssl verify filetries to verify using only the leaf cert which NEVER works and gives no useful evidence; that's why
-untrustedwith the chain(s) is suggested. Try
openssl s_client -showcertsand look at all the received certs, and the verification callback log. @GregAskew+ note OpenSSL never uses AIA to fetch chain, and by default does not check revocation at all; that must be requested by the calling code and I have no idea if 389 does or can.