Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

The question I have may be more of principle than anything else, but here's my dilemma.

I manage an email system for a small company (about 20 email users). We own a plain-letter .com domain name through Network Solutions. Our email service is hosted by Google Apps.

Recently (Feb. 2011) we've been having customers report that they aren't getting our emails. Upon further investigation it seems that the failed emails are all to a common (well known) domain. We have not received any bounce messages for the emails. We've also contacted a few of the intended recipients, who have reported that the messages are not in their spam box; they simply did not receive anything. In these cases we re-sent the same email to an alternate address on another domain, which was successful received.

One customer contacted their email provider about the issue. The provider recommended that we submit a form to be white-listed by their domain.

Here's where my problem begins. I feel like this is heading down a slippery slope. Doesn't this undermine the very principle of email? If this is the appropriate action to take in these situations where will it end? In theory (following this model) it could be argued that eventually one will first need to "whitelist" (or more appropriately termed "authenticate") themselves with an email host before actually sending any messages. More to this point, what keeps the "bad" spammers from doing the same thing...? We've just gone full circle.

I know avoiding anti-spam measures is a big cat-and-mouse game, but I think this is the wrong way of "patching" the problem. Email standards say that messages should not just disappear silently. I have a problem supporting a model that says "you must do < this > to make sure your emails aren't ignored".

I have a notion to call the provider and voice my complaint, although I have a feeling it will probably fall on deaf ears. Am I missing something here? Is this an acceptable approach to email spam problems? What should I do?

share|improve this question
add comment

3 Answers 3

up vote 10 down vote accepted

I find far too many legitimate sites are shooting themselves in the foot. They (unwittingly I hope) configure their servers in such a manner that they appear to forging their identity. Common problems include:

  • Missing or incorrect PTR record.
  • A record for the PTR does not point back to the originating IP address.
  • The name used in the HELO command has an invalid top level domain like local, lan, or localdomain.
  • The name used in the HELO command does not have an A record.
  • The A record for the name in the HELO command does not return the IP address being used.
  • No SPF record for the domain being used in the HELO command.
  • No SPF configured for the domain of the envelope sender.
  • SPF for the domain of the envelope sender prohibits use of the IP address being used to send the email.
  • Domain is not listed with DNSWL.org.
  • Configuring the address of the mail server as a second level domain (example.com) rather than a third or forth level domain (mail.example.com).
  • Not accepting mail to postmaster.

Avoid all of the above, and you will have a much better chance of getting your mail delivered.

EDIT: I have found Port25 Solutions Inc. has a very good automated verification service listed on their E-mail Authentication page. Many thanks to them for their fine service. It is designed to verify DKIM signatures, but gives excellent feedback for most of the items listed above. Check in the Port25 resources section, and use the appropriate email address to get the results mailed to your desired e-mail address. Remember if you need to do DNS changes it can take a day or so to be reflected in all the caches. Worst case should be two times your Time to Live setting.

share|improve this answer
    
+1 for a nice list, thanks! –  jftuga Feb 28 '11 at 18:34
    
+1 for a good list of stuff to check from the technical side, but if you've taken care of all of that already you're back up Ye Olde Creek :-) –  voretaq7 Feb 28 '11 at 19:04
    
+1 for the great check-list. Since Google hosts our email I just assumed they take care of the PTR/SPF records. I will check into this and make sure I haven't unknowingly neglected any important details. –  joxl Feb 28 '11 at 19:20
    
@joxl I added a note about Port25's verification resource. I find it extremely useful to verify my server's configuration. –  BillThor Feb 28 '11 at 21:17
add comment

There's not much anyone here can say to help you -- your problem isn't technical as you've mentioned: It's a matter of principle.

Unfortunately email in the 21st century is very much a "You want to talk to my users, you play by my rules" thing. If you want to resolve this issue you need to submit yourself to the remote site's whitelisting procedures.

I don't necessarily agree with this myself -- It does go against the spirit of mutual cooperation that makes email (and the internet in general) work -- but given the fact that roughly 50% of the mail that comes to my domains is spam and gets discarded by filtering I do understand it. In fact when we get complaints that clients aren't able to send email to us we follow similar procedures in gathering the valid list of sending servers and whitelisting their domain. It's not pretty, but it keeps the volume of junk in everyone's inbox manageable.

As a consolation prize I offer you a copy of the email lecture that I give to new support techs when they send me their first "X says Y can't get email from them" -- It's certainly not something you can tell your clients/users, but it may give you a laugh during the dark times of email troubleshooting. Feel free to embellish as appropriate :-)

EMail is not a reliable delivery system.  There is no guarantee that any message
sent by party A will be received by its intended recipient, or by anyone at all.

EMail depends upon the cooperation of the sender & recipient's servers, as well
as potentially dozens of other servers that will handle your message along the way.
Each server has its own standards for what is or is not an acceptable message, and
may delay or discard your email for any reason, or for no reason at all, and they
probably will not tell you (or your sysadmin) what they're doing.

If your correspondence is time-sensitive or critically important email may not be
the best medium. Consider a telephone call, or if sending lots of documents
FedExing a CD.
share|improve this answer
1  
+1 for the consolation prize! We've got a similar standard e-mail, plus that there's three 'worst offenders' and if the destination is one of them pick another medium. –  Chris S Feb 28 '11 at 16:14
    
Added to our knowledgebase. I've had to explain more than once that E-mail is not instant messaging. –  Koos van den Hout Feb 28 '11 at 18:30
    
+1 for the consultation in general, and the prize rocks! It really did shine a warm light into my "dark" day :) I've checked into submitting the whitelist form and in the the meantime learned that the "offending" email host claims to bounce all blocked emails. I think it's interesting that we haven't gotten any bounce messages and therefore am doing a little more research first. –  joxl Feb 28 '11 at 19:14
1  
@joxl - Best defense there is to give them the segment of your mail server logs that show their server accepting (with a 2xx code & message ID) your mail. Then you can definitively say they lost the message after accepting it for delivery :) –  voretaq7 Feb 28 '11 at 19:44
add comment

I understand your frustration but I also understand why an ever increasing number of systems are going a bit overkill on spam. That's the way things are and, whether we like it or not, we simply have to live with it. Spam is a serious problem that's wasting vast sums of money.

In the majority of cases you won't need to request a whitelisting if your own system is well configured. By well configured I mean having things that may not actually be required by any standard but are expected based on common usage. Things such as proper SPF records, DKIM and/or DomainKeys, etc.

As for the debatable issue of systems quietly swallowing emails that are identified as spam, it's a sad reality that if those messages are actively rejected they very frequently result in a large increase of spam from the same source. Current needs dictate that the standards sometimes have to be violated, simply because there are too many people prepared to abuse the systems.

share|improve this answer
    
There are standards and best practices. If you follow them you run little risk of having your email classified as Spam. Most Spam clearly violates standards and best practices. Unfortunately, there are standards I can not enforce because too many legitimate senders violate them. –  BillThor Mar 5 '11 at 19:17
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.