What problems arise when we use a self-signed certificate for the SMTP protocol, that is, when a SMTP server uses a self-signed certificate?

As long as the user accepts the exception warning due to self-signed certificate (which can be the case in the Thunderbird mail client), I would think there is no problem.

Can anyone tell me what other problems may arise?

| improve this question | | | | |

On server to server communications there's no problem, because SMTP servers accepts any certificate and even falls back to unencrypted connection. In that sense, there's no public key infrastructure for MX servers. Against downgrade attacks there's DNS-Based Authentication of Named Entities (DANE) (see RFC 7672 on SMTP Security via Opportunistic DANE TLS), utilizing public keys published in DNS over public keys signed by a CA. Self-signed certificates are ok for that.

For client to server communications it's at least more convenient to have a trusted CA signed certificate. If users learn to accept an exception, it's way more likely they do so even when there's a MitM attack in place. Moreover, e.g. Android mail client only allows requiring a trusted certificate or accepting any, making the MitM attack even easier. That's why I would recommend using a CA signed certificate for client communications.

It's even possible to get a free Let's Encrypt certificate for your SMTP/IMAP server as it's not limited to use the same certificate for multiple protocols. This pretty much removes any need for self-signed certificates with publicly-referenced mail servers. (Example configuration for Postfix.)

| improve this answer | | | | |
  • 3
    "SMTP servers accept any certificate" - that sounds like a horribly insecure default setting.. – WooShell Oct 28 '19 at 14:44
  • 4
    @WooShell Consider that emails commonly have to transverse multiple SMTP hops to get from A to B, many which may not be secure or even owned by the sender or receiver. The spec makes no attempts at enforcing encryption on every hop. The only way to faithfully encrypt and verify emails is to handle that on top of SMTP e.g. encrypt/sign the message using something like SMIME via PGP/GPG or trusted certs signed by a trusted CA. – Mark Lopez Oct 28 '19 at 14:52
  • 1
    If it was a design, it would be horrible, indeed. But email wasn't designed encryption in mind back then, and we have kept it backwards compatible. That's why the current philosophy is that any encryption is considered better than the default plain text. The DANE is a solution that enables both backwards compatibility and strict policies. – Esa Jokinen Oct 28 '19 at 15:24
  • Be aware that some email clients/apps REALLY don't want to use a self-signed certificate, and make it difficult to accept one. I eventually gave up and bought a cheap $9 certificate just to avoid the issue. – simpleuser Oct 29 '19 at 19:08

A self signed certificate might deter trivial attempts to intercept mail, but would be of limited value as they can be MITM'd. If you do want to go this route you should create a CA and have your users accept the CA. Unfortunately this is possibly a bad idea for your users (as you could sign rogue certs), and of limited value because the traffic could still be intercepted between other MTAs and yours.

Something no one else has commented on - while encrypted delivery is not part of the SMTP protocol, some providers (eg GMAIL) will mark the email as having been encrypted or not - and I speculate (I don't use gmail, so don't know for sure) that MTAs using self signed certs won't be considered as secure.

As others have mentioned, using LetsEncrypt is generally not to big an ask, and a better solution.

| improve this answer | | | | |

The same problem that you have with every self-signed certificate, client can not and must not trust the server. If you are the only client for your smtp server you can add the CA and not even have to accept the exception. If i am one of the users of your server i will definitely not use it. There is no guaranty that i am connecting to the right server, i will assume its a rogue server.

P.S still encryption part is OK, traffic is encrypted

| improve this answer | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.