How to decrypt TLS traffic with Wireshark using RSA asynchronous encryption?

Question

I want to be able to capture and decrypt TLS traffic that one off my internal application (that I don't have access) makes to the internet. (For testing I am using Postman to create a request to a secure server.)

1. What I did was to add an Nginx as a reverse proxy.

2. I created a self-signed certificate with the next command:

   sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout localhost.key -out localhost.crt
   

3. I added the ssl configuration to the /etc/nginx/sites-enabled/default file the certificate.

   server {
   
        # SSL configuration
        #
        listen 443 ssl default_server;
        ssl_certificate /home/mavi/nginx/keys/localhost.crt;
        ssl_certificate_key /home/mavi/nginx/keys/localhost.key;
        ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                #try_files $uri $uri/ =404;
                proxy_pass https://myService.com;
                proxy_http_version 1.1;
        }
   }
   

4. I added the key that I generated with OpenSSL in Wireshark Edit> Preferences > SSL > RSA Keys list.

5. I create the request pointing to my proxy (HTTPS://127.0.0.1) and it redirects the request to the external service and I get a correct response.

6. I captured the traffic made by the previews request but Wireshark it's not able to decrypt the traffic.

Why am I not able to decrypt the data if I am using an RSA key exchange?

I have been investigating (please correct me if I am wrong) the types of asynchronous encryption. I know that RSA sends the public key with its certificate to the client and then the client uses this public key to encrypt the data. Then when this encrypted data is sent to the server, it uses the private key to decrypt the information. So if I use this private key in Wireshark I should be able to decrypt the data, right?

Also, I have read that for security we have stopped using RSA and now we moved to use Diffie Hellman algorithms (ECDH, ECDHE DFH). Diffie Hellman and its variants don't use the private key anymore, it uses random session keys that it saves in RAM. (As a note to decrypt Diffie Hellman packages we need to add a global variable in the system $SSLKEYLOGFILE and import this file to Wireshark. This only works with Google Chrome, Firefox, CURL not for internal apps)

So I think that is why I am not able to decrypt the data. Because in the cipher's negotiation from my client and my server (both in my local computer) are using Diffie Hellman as the Asynchronous encryption even when I create an RSA certificate. I am guessing they are using Diffie Hellman because in the TLS handshake I see that they agree to use 0xc030

If I'm right then I should change my question to How can I specify the ciphers to use in Nginx configuration? Right now my Nginx has the ssl_prefer_server_ciphers on; configuration.

I will add the TLS handshake so you can see the cipher's negotiation. Client Hello

Server Hello

Cipher Renegotiation

Answer 1

TLS 1.2 with ECDHE-RSA-AES256-GCM-SHA384 has forward secrecy. Notice its presence on the intermediate level of Mozilla's cipher lists. Inability to decrypt with only the server key is a feature.

Extract the client keys as well. As this terminates TLS on nginx, get nginx's client keys. Absent any easy option to enable this, compile and load your own hooks. See for example Extracting openssl pre-master secret from nginx which makes use of someone's sslkeylog.c. Because OpenSSL's keylog API is known, such a library works for any client, including nginx with OpenSSL.

As usual, be careful of security and operational concerns when compiling third-party code. Should nginx not be a requirement, consider some other proxy where key logging is a feature like mitmproxy

Answer 2

As I was guessing the issue was in the ciphers that the server and the client agreed.

We can only decrypt TLS/SSL packet data if RSA keys are used to encrypt the data. If a Diffie-Hellman Ephemeral (DHE) or RSA ephemeral cipher suite is used, the RSA keys are only used to secure the DH or RSA exchange, not encrypt the data. Thus, even if you have the correct RSA private key, you will not be able to decrypt the data with ssldump, Wireshark, or any other tool. You can check which cipher suite is being used by examining the Server Hello packet sent by the host that holds the private key, if the cipher suite specified begins TLS_DHE or SSL_DHE, you will not be able to decrypt the data. If the server sends a ServerKeyExchange message, you will not be able to decrypt the data. For DHE, your only option (if possible) is to modify either the client or server configuration so that DHE cipher suites are not used.

ECDHE or DH (Diffie-Hellman), both create session keys that only the entities involved in the SSL connection can access. Because the session keys are not linked to the server’s key pair, the server’s private key alone cannot be used to decrypt any SSL session.

All TLS_RSA cipher suites have been marked as WEAK because they don't provide forward secrecy, which means that in TLS_RSA private key is used to decrypt the data: if the private key gets compromised in the future, all recorded traffic can be decrypted using it.

Now answering my own question: How can I specify the ciphers to use in Nginx configuration?

I had to configure my Nginx server to use only TLS_RSA ciphers in the /etc/nginx/nginx.conf file

http {

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;
        ssl_ciphers AES256-SHA256:AES256-SHA:AES128-SHA256:AES128-SHA:RC4-SHA:RC4-MD5:DES-CBC3-SHA;
}

That way I am able to decrypt the data. As we can see in the image below the client and the server agreed to use TLS_RSA_WITH_AES_256_CBC_SHA(0X0035) and Wireshark could decrypt the data using the private key.