This is what worked for me. I would however appreciate any comments or answers that can expand on what some of these choices actually mean!
Open the MMC console on the Remote Desktop server you want to generate the certificate for, and add the Certificates snap-in, selecting the "Computer account" and "Local computer" options. Go to Personal/Certificates, right-click and select All Tasks -> Advanced Operations -> Create Custom Request.
Click Next. Select "Proceed without enrollment policy" and click Next again.
For Template, I chose "(No template) CNG key". I have found some posts saying that you need to choose the Legacy option instead, but I don't see any reason why this would be necessary and indeed the CNG option worked as expected.
For Request format, I chose PKCS #10.
In the Certificate Information dialog, click on Details and then Properties.
In the General tab, add a friendly name and a description.
In the Subject tab, add the fully-qualified DNS name of the server (or server farm) as the "Common name". Note that a certificate request with an unqualified name (whether as the subject or as an alternative name) is likely to be rejected by the certificate signing authority.
I also added Organization, Locality, State, and Country. If the server has more than one DNS name, you may also wish to add the alternative names at this point.
In the Extensions tab, under Extended Key Usage, add Server Authentication. I did not make any other changes in this tab. (Some posts say you should also include Code Signing, presumably so that you can sign RDP files; this does not appear to be necessary, as I was able to use
rdpsign to sign my RDP file and the certificate was accepted by the Microsoft client.)
In the Private Key tab, under Key options, I changed the key size to 2048 and set the flag for "Make private key exportable". This is necessary because the Deployment Properties dialog will only allow you to import a certificate as a file, the certificate and private key will then be transferred to the session host server(s). I did not make any other changes under this tab.
After dismissing the Properties dialog, click Next. Save the request as a file in Base64 format. Click Finish. Submit the request to your certificate authority, and once the new certificate has been generated, download it into a .crt file.
In the MMC console, right-click on Personal/Certificates and select All Tasks -> Import. Select the response file and click Next. Check that the Personal store is selected and click Next. Click Finish. Click OK when you get the message saying the import was successful.
Double-click the new certificate to open it. Check that the certificate is shown as valid; if not, you may need to import an intermediate CA certificate provided by your certificate authority. Also check that the certificate shows the message "You have a private key that corresponds to this certificate" on the General tab.
Right-click on the new certificate and select Export. Click Next. Select the option to export the private key and click Next.
In the Export File Format dialog, PKCS#12 was the only available choice; I used the default settings, i.e., I left the "include all certificates in the certification path if possible" option checked and I left all the other options unchecked. Click Next. (Edit: in Server 2019 the "Enable certificate privacy" option is also enabled by default, this is described here and as far as I can see you may as well leave it enabled.)
Choose the option to protect the private key using your user account and click Next. Enter a file name and click Next. Click OK when you get the message saying the export was successful. (Note that the file will be saved by default to the same location you imported the certificate file from.)
Going back to Server Manager and the Deployment Properties wizard, select the "RD Connection Broker - Enable Single Sign On" option and click "Select existing certificate". Choose the exported .pfx file and select the mandatory "Allow the certificate to be added to the Trusted Root Certification Authorities certificate store on the destination computers" option. Click OK. Click Apply.
NB: in order for the certificate to be used when clients connect, you must install it for the "Enable Single Sign On" option rather than, as I originally assumed, the "Publishing" option. (You don't have to actually use SSO, you can configure whether SSO is attempted via group policy at the client end.)
As described here, you can most easily check what certificate is being served to clients by connecting via the IP address of the server rather than the name.
Configuring certificates and single sign-on, especially the section titled "Common Mistakes Creating Certificates".
Securing RDS with certificates.