Yesterday the IIS on our build server (running Windows Server 2012) started refusing our clients' certificates. The certificates are signed using our own self-signed CA cert that has been added to Trusted Root Certification Authorities (local machine). This has been working flawlessly until yesterday. I've desperately been trying to find out what might have changed that could cause this. I see no Schannel errors or warnings in the Event Viewer.
However, after running openssl against the server I noticed something suspicious. It looks like the IIS is not sending a single CA in its list of trusted client cert authorities. The log looks like this:
CONNECTED(00000144)
depth=0 CN = Localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = Localhost
verify return:1
---
Certificate chain
0 s:/CN=Localhost
i:/CN=Localhost
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC+zCCAeOgAwIBAgIQOkacw1RkE4tI9+HnyEXFvzANBgkqhkiG9w0BAQsFADAU
MRIwEAYDVQQDEwlMb2NhbGhvc3QwHhcNMTMwODA1MDgwOTU1WhcNMzkxMjMxMjM1
OTU5WjAUMRIwEAYDVQQDEwlMb2NhbGhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQC/kc5BLMcmuNoZe8jkrJQt/kZFD7EnVOtvEEJt0dZJG008TqXD
MdXnybBWPCbvQIFoxREY6wjPExcU39SzbCWLGV99Z+eR0zFkOpK3SSppe9fulkP7
ktiDWTSkgJUx1/EpHeJHL1hy7YKRFYOtPlewZYjaklh/wND5F88mOri/lEoENpWO
0fLrJS+Nnizeti7LEzstNtU7+AH4h6njCujrQwjwdCr1QTggjLj3iOy7fpUqYwKe
mNGNIAR8XI06JzYAFDpcdo4PMZScNfd0cqcMIHJuWUoaciW9qwrbHWyr1B3hBCX0
luQSF4uHVbT+8yOI4fOWL4PTL/6ZNEfl4WrxAgMBAAGjSTBHMEUGA1UdAQQ+MDyA
EHhoR/6NVn2yfadGy1PvZ26hFjAUMRIwEAYDVQQDEwlMb2NhbGhvc3SCEDpGnMNU
ZBOLSPfh58hFxb8wDQYJKoZIhvcNAQELBQADggEBAIujtVAr3UvG7dB55SBgQP5p
AiOum0DM9xULarl+Wz/GdTvdK65PcUB34DlG8pEhz5nRsX5I/nZvLF/7U5OCICp2
Gnvbm2jLYnlacB16+ds/4cgG65a/CddSdVyRIYa2YdGXZGiJ6zTkEQWEH4tXmkO+
InzHsBEVO1MT1nAfkZp6MzgEbCv8Xus3QIxdnJZZYHMzXcD+48oQEfP5BhHXW/iN
MlNsuN8wwwpS61r2g9Bu8AhMcbnvoMNdYbBtPC5+ltlOQK0RNNTcqOr4kJj/BwO3
fGS8/lh9FTZFq8c4ES94hoEu4szUfA4jkTvt9SWossOBPehhIWKUgx5MIdC6Hgc=
-----END CERTIFICATE-----
subject=/CN=Localhost
issuer=/CN=Localhost
---
No client certificate CA names sent
---
SSL handshake has read 1291 bytes and written 487 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-SHA256
Session-ID: C1480000D74420B9A5C00326C73B6ACC652ED4D077CD02C72CE347CE2F603CA8
Session-ID-ctx:
Master-Key: F8E3625F2A36FE2CA963F2FE2A0774B7B6AEEC0D0592DC9CD46C5FC98ADECD77
82FE8CF91D71C318A970BEEA4BE384A8
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1377623899
Timeout : 300 (sec)
Verify return code: 18 (self signed certificate)
---
read:errno=10054
---
Certificate chain
0 s:/CN=Localhost
i:/CN=Localhost
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC+zCCAeOgAwIBAgIQOkacw1RkE4tI9+HnyEXFvzANBgkqhkiG9w0BAQsFADAU
MRIwEAYDVQQDEwlMb2NhbGhvc3QwHhcNMTMwODA1MDgwOTU1WhcNMzkxMjMxMjM1
OTU5WjAUMRIwEAYDVQQDEwlMb2NhbGhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQC/kc5BLMcmuNoZe8jkrJQt/kZFD7EnVOtvEEJt0dZJG008TqXD
MdXnybBWPCbvQIFoxREY6wjPExcU39SzbCWLGV99Z+eR0zFkOpK3SSppe9fulkP7
ktiDWTSkgJUx1/EpHeJHL1hy7YKRFYOtPlewZYjaklh/wND5F88mOri/lEoENpWO
0fLrJS+Nnizeti7LEzstNtU7+AH4h6njCujrQwjwdCr1QTggjLj3iOy7fpUqYwKe
mNGNIAR8XI06JzYAFDpcdo4PMZScNfd0cqcMIHJuWUoaciW9qwrbHWyr1B3hBCX0
luQSF4uHVbT+8yOI4fOWL4PTL/6ZNEfl4WrxAgMBAAGjSTBHMEUGA1UdAQQ+MDyA
EHhoR/6NVn2yfadGy1PvZ26hFjAUMRIwEAYDVQQDEwlMb2NhbGhvc3SCEDpGnMNU
ZBOLSPfh58hFxb8wDQYJKoZIhvcNAQELBQADggEBAIujtVAr3UvG7dB55SBgQP5p
AiOum0DM9xULarl+Wz/GdTvdK65PcUB34DlG8pEhz5nRsX5I/nZvLF/7U5OCICp2
Gnvbm2jLYnlacB16+ds/4cgG65a/CddSdVyRIYa2YdGXZGiJ6zTkEQWEH4tXmkO+
InzHsBEVO1MT1nAfkZp6MzgEbCv8Xus3QIxdnJZZYHMzXcD+48oQEfP5BhHXW/iN
MlNsuN8wwwpS61r2g9Bu8AhMcbnvoMNdYbBtPC5+ltlOQK0RNNTcqOr4kJj/BwO3
fGS8/lh9FTZFq8c4ES94hoEu4szUfA4jkTvt9SWossOBPehhIWKUgx5MIdC6Hgc=
-----END CERTIFICATE-----
subject=/CN=Localhost
issuer=/CN=Localhost
---
**No client certificate CA names sent**
---
SSL handshake has read 1291 bytes and written 556 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-SHA256
Session-ID: C1480000D74420B9A5C00326C73B6ACC652ED4D077CD02C72CE347CE2F603CA8
Session-ID-ctx:
Master-Key: F8E3625F2A36FE2CA963F2FE2A0774B7B6AEEC0D0592DC9CD46C5FC98ADECD77
82FE8CF91D71C318A970BEEA4BE384A8
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1377623899
Timeout : 300 (sec)
Verify return code: 18 (self signed certificate)
---
Notice the text: No client certificate CA names sent. When I debug it using our Java client I seem to get the same issue. During the handshake it says: "Cert Authorities: ".
My understanding was that IIS should return all certs in Trusted Root Certification Authorities. Running the same request against the IIS at my local dev machine confirms that. That IIS server returns a ton of certs (including our self signed CA cert).
So my question is: why is IIS no longer returning any trusted CA certs during the handshake?
Update 1 I have found some more info by activating detailed CAPI logging.
- UserData
- CertGetCertificateChain
- Certificate
[ fileRef] 4FEA293C62EAF436D286F700F618814E72D49347.cer
[ subjectName] lIv-zQE|3M-OywU
- AdditionalStore
- Certificate
[ fileRef] 4FEA293C62EAF436D286F700F618814E72D49347.cer
[ subjectName] lIv-zQE|3M-OywU
- ExtendedKeyUsage
- Usage
[ oid] 1.3.6.1.5.5.7.3.2
[ name] Client Authentication
- Flags
[ value] 40000004
[ CERT_CHAIN_CACHE_ONLY_URL_RETRIEVAL] true
[ CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT] true
- ChainEngineInfo
[ context] machine
- CertificateChain
[ chainRef] {317A4B99-2193-4AA6-9D3D-768AF747C66D}
- TrustStatus
- ErrorStatus
[ value] 1010040
[ CERT_TRUST_REVOCATION_STATUS_UNKNOWN] true
[ CERT_TRUST_IS_OFFLINE_REVOCATION] true
[ CERT_TRUST_IS_PARTIAL_CHAIN] true
- InfoStatus
[ value] 0
- ChainElement
- Certificate
[ fileRef] 4FEA293C62EAF436D286F700F618814E72D49347.cer
[ subjectName] lIv-zQE|3M-OywU
- SignatureAlgorithm
[ oid] 1.2.840.113549.1.1.11
[ hashName] SHA256
[ publicKeyName] RSA
- PublicKeyAlgorithm
[ oid] 1.2.840.113549.1.1.1
[ publicKeyName] RSA
[ publicKeyLength] 2048
- TrustStatus
- ErrorStatus
[ value] 1000040
[ CERT_TRUST_REVOCATION_STATUS_UNKNOWN] true
[ CERT_TRUST_IS_OFFLINE_REVOCATION] true
- InfoStatus
[ value] 4
[ CERT_TRUST_HAS_NAME_MATCH_ISSUER] true
- ApplicationUsage
[ any] true
IssuanceUsage
- RevocationInfo
- RevocationResult The revocation function was unable to check revocation because the revocation server was offline.
[ value] 80092013
- EventAuxInfo
[ ProcessName] lsass.exe
- CorrelationAuxInfo
[ TaskId] {11C0F7E0-B3E6-4B4B-AA98-9A2AE7800A03}
[ SeqNumber] 3
- Result A certificate chain could not be built to a trusted root authority.
[ value] 800B010A