Email delivery management grievances

Question

The question I have may be more of principle than anything else, but here's my dilemma.

I manage an email system for a small company (about 20 email users). We own a plain-letter .com domain name through Network Solutions. Our email service is hosted by Google Apps.

Recently (Feb. 2011) we've been having customers report that they aren't getting our emails. Upon further investigation it seems that the failed emails are all to a common (well known) domain. We have not received any bounce messages for the emails. We've also contacted a few of the intended recipients, who have reported that the messages are not in their spam box; they simply did not receive anything. In these cases we re-sent the same email to an alternate address on another domain, which was successful received.

One customer contacted their email provider about the issue. The provider recommended that we submit a form to be white-listed by their domain.

Here's where my problem begins. I feel like this is heading down a slippery slope. Doesn't this undermine the very principle of email? If this is the appropriate action to take in these situations where will it end? In theory (following this model) it could be argued that eventually one will first need to "whitelist" (or more appropriately termed "authenticate") themselves with an email host before actually sending any messages. More to this point, what keeps the "bad" spammers from doing the same thing...? We've just gone full circle.

I know avoiding anti-spam measures is a big cat-and-mouse game, but I think this is the wrong way of "patching" the problem. Email standards say that messages should not just disappear silently. I have a problem supporting a model that says "you must do < this > to make sure your emails aren't ignored".

I have a notion to call the provider and voice my complaint, although I have a feeling it will probably fall on deaf ears. Am I missing something here? Is this an acceptable approach to email spam problems? What should I do?

Answer 1

I find far too many legitimate sites are shooting themselves in the foot. They (unwittingly I hope) configure their servers in such a manner that they appear to forging their identity. Common problems include:

• Missing or incorrect PTR record.
• A record for the PTR does not point back to the originating IP address.
• The name used in the HELO command has an invalid top level domain like local, lan, or localdomain.
• The name used in the HELO command does not have an A record.
• The A record for the name in the HELO command does not return the IP address being used.
• No SPF record for the domain being used in the HELO command.
• No SPF configured for the domain of the envelope sender.
• SPF for the domain of the envelope sender prohibits use of the IP address being used to send the email.
• Domain is not listed with DNSWL.org.
• Configuring the address of the mail server as a second level domain (example.com) rather than a third or forth level domain (mail.example.com).
• Not accepting mail to postmaster.

Avoid all of the above, and you will have a much better chance of getting your mail delivered.

EDIT: I have found Port25 Solutions Inc. has a very good automated verification service listed on their E-mail Authentication page. Many thanks to them for their fine service. It is designed to verify DKIM signatures, but gives excellent feedback for most of the items listed above. Check in the Port25 resources section, and use the appropriate email address to get the results mailed to your desired e-mail address. Remember if you need to do DNS changes it can take a day or so to be reflected in all the caches. Worst case should be two times your Time to Live setting.

Answer 2

There's not much anyone here can say to help you -- your problem isn't technical as you've mentioned: It's a matter of principle.

Unfortunately email in the 21st century is very much a "You want to talk to my users, you play by my rules" thing. If you want to resolve this issue you need to submit yourself to the remote site's whitelisting procedures.

I don't necessarily agree with this myself -- It does go against the spirit of mutual cooperation that makes email (and the internet in general) work -- but given the fact that roughly 50% of the mail that comes to my domains is spam and gets discarded by filtering I do understand it. In fact when we get complaints that clients aren't able to send email to us we follow similar procedures in gathering the valid list of sending servers and whitelisting their domain. It's not pretty, but it keeps the volume of junk in everyone's inbox manageable.

As a consolation prize I offer you a copy of the email lecture that I give to new support techs when they send me their first "X says Y can't get email from them" -- It's certainly not something you can tell your clients/users, but it may give you a laugh during the dark times of email troubleshooting. Feel free to embellish as appropriate :-)

EMail is not a reliable delivery system.  There is no guarantee that any message
sent by party A will be received by its intended recipient, or by anyone at all.

EMail depends upon the cooperation of the sender & recipient's servers, as well
as potentially dozens of other servers that will handle your message along the way.
Each server has its own standards for what is or is not an acceptable message, and
may delay or discard your email for any reason, or for no reason at all, and they
probably will not tell you (or your sysadmin) what they're doing.

If your correspondence is time-sensitive or critically important email may not be
the best medium. Consider a telephone call, or if sending lots of documents
FedExing a CD.

Answer 3

I understand your frustration but I also understand why an ever increasing number of systems are going a bit overkill on spam. That's the way things are and, whether we like it or not, we simply have to live with it. Spam is a serious problem that's wasting vast sums of money.

In the majority of cases you won't need to request a whitelisting if your own system is well configured. By well configured I mean having things that may not actually be required by any standard but are expected based on common usage. Things such as proper SPF records, DKIM and/or DomainKeys, etc.

As for the debatable issue of systems quietly swallowing emails that are identified as spam, it's a sad reality that if those messages are actively rejected they very frequently result in a large increase of spam from the same source. Current needs dictate that the standards sometimes have to be violated, simply because there are too many people prepared to abuse the systems.