Why don’t SMTP servers just require all senders to be authenticated before accepting mail? [closed]

Question

Why don’t SMTP servers just require all senders to be authenticated before accepting mail?

Answer 1

The mail submission protocol (RFC4409) does exactly what you ask (it requires both encryption and authentication) and is used by most ISPs, but it's probably not what you mean as it's outbound-only.

For inbound it's harder. There can't be a user/pass mechanism because any server in the world could be sending you something legitimately without advance notice, so the receiving server needs to investigate the sender. There are several protocols to allow this to work within SMTP, in particular: SPF, DKIM and DMARC. SPF has a chequered history that wasn't helped by Microsoft's SenderID protocol that really confused things and wasn't helpful.

SPF sets out to authorise sources of email via IP address or hostname using DNS entries. Unless rules result in outright PASS or FAIL results, it's not really effective, and imposes limits on user settings (e.g. preventing users from using ISP's mail servers to send from their company's domain). Its effectiveness is limited, however it is supported by many larger ISPs.

DKIM sets out to prove that message contents has not been tampered within transit by providing a cryptographic hash of messages headers and bodies. It has parallels with S/MIME signatures, but at a different level in the stack.

SPF and DKIM work well together, but they lack management oversight as to what to do on failure - no mechanismss are provided to report contraventions, and most failures just end up lost in log files. DMARC sets out to solve that by defining reporting mechanisms. As such, DMARC removes the guesswork as to what to do with SPF and DKIM failures that is otherwise required.

In answer to your question, SPF is easy to implement (it's just a DNS record), but deciding what to put in it can be messy. DKIM is complicated and integration with mail servers can be difficult. DMARC relies on both of them being implemented. Taken together, that's why many domains do not implement any of them.

There are possible alternatives like DJB's IM2000 protocol which shifts the burden of storage and authentication to the sender, but it would involve replacing the entire world of SMTP servers, so it's more an academic exercise than anything else.

Answer 2

What problem are you trying to solve?

Let's say every SMTP server required a username and password. You would now have to maintain a username and password for every person that might ever send email to any of your users. That's not possible. Imagine, however, that it is possible. A spammer would then simply create a new username and password and send you spam. If you disabled the account, they would create a new username and password. It is no better than if you blocked their IP address; they move to a new IP address.

Let's say you only accepted email that was GPG signed so that you knew who sent the message. That's another form of authentication but it doesn't require setting up a username and password for every user in the world. Spammers would simply sign all their spam. That would slow them down a little, and you would know that, yes, this spam absolutely came from a particular person. However you would still have to block that person, who would then generate a new GPG key, and the cycle would repeat.

SMTP is a "federated service". Every email system is self-contained but the federation (the collection of every SMTP server on the internet) can communicate with each other. The problem with such a large federation is that it has to work on trust; and spammers break that trust. Trust is a social issue. You can't solve social issues with technology.