I have a Server 2012 RD farm that works fine if configured with the self-signed certs generated by Server Manager, but not with certs from our internal CA.
With the self-signed certs, our remote clients can connect but of course get security warnings due to the untrusted certificates. The clients do trust our root CA, so we should be able to eliminate those by using certs from our internal CA instead.
However, when I do configure the farm to use certs from our internal CA (which runs AD CS on Server 2008 R2,) clients can log into the RD Web site but cannot open RDP sessions. They get errors such as the following on Windows 7 (none of the clients are newer than 7 so I haven't tried 8) :
Your computer can't connect to the remote computer because an error occurred on the remote computer that you want to connect to. Contact your network administrator for assistance.
Your computer can't connect to the remote computer because the Remote Desktop Gateway and the remote computer are unable to exchange policies. This could happen due to one of the following reasons:
- The remote computer is not capable of exchanging policies with the Remote Desktop Gateway.
- The remote computer's configuration does not permit a new connection.
- The connection between the Remote Desktop Gateway and the remote computer ended.
Contact your network administrator for assistance.
In turn these will appear in the server logs:
(Source: Schannel; Event ID: 36874) [sic] An TLS 1.2 connection request was received from a remote client application, but none of the cipher suites supported by the client application are supported by the server. The SSL connection request has failed.
(Source: Schannel; Event ID: 36888) A fatal alert was generated and sent to the remote endpoint. This may result in termination of the connection. The TLS protocol defined fatal error code is 40. The Windows SChannel error state is 1205.
Our root CA and issuing CA are both using SHA512 hashes and 4096-bit public keys. I notice that the self-signed certs from Server Manager that do work are using SHA256 and 2048-bit keys, so I'm wondering if the stronger encryption is not supported by RDP in Windows 7.
(I can't test this easily as I can't make our CA hand out a cert using SHA256, I'm guessing because the CA's own public key is too big. Even if it did, the client would still need SHA512 to validate the issuing CA against the root CA.)
The strange thing is that it does work with our certs except for "RD Connection Broker - Enable Single Sign On." If I leave that one set to the self-signed cert but use ours for the other three, everything works basically fine (apart from users having to enter their password three times.)
In that case, Internet Explorer on the client trusts one of our certs without issue even though it has SHA512. This makes it seem rather odd that the stronger crypto would trip up RDP - I would have assumed they'd both use a provider built into Windows.