Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I just added a new DNSBL to the list of ones that I use on my small mailserver. I set it up to be first on the list because I wanted to see what it missed but other lists caught.

I'm watching mail logs out of the corner of my eye while I do other things, just to make sure that things looked sane, and I started wondering about how to measure the relative effectiveness of various lists. I suppose I could give each list a week at the top of the heap and then measure how much the other lists catch that week, but that seems like a real hack.

Are there any tools or techniques for measuring the relative effectiveness of different blocklists on a specific server? I can measure false positives by user complaint volume, so my concern is addresses that one blocklist misses but another catches.

share|improve this question

3 Answers 3

If all you want to do is determine "which is the minimal set of blacklists I should subscribe to in order to catch the maximum amount of spam", then what you really want to do is:

  • Tally all of your blocked spam by IP address (so analyse your mail logs to get a table of blocked IPs, with the count of how many spams came from each, perhaps)
  • Run all those IPs through all of your blacklists (trivial to do, just needs a quick DNS lookup for each IP)
  • Score each blacklist against how many of the IPs (and how many spams came from each IP) each blacklist caught.

This is complicated by the time-varying nature of the contents of each of the blacklists, so perhaps having a logcheck-type process (running regularly, analysing new log entries) is the way to go, which can then get reported back to you as required.

All of this isn't all that tricky to rig up with a few lines of shell or $SCRIPTING_LANGUAGE_OF_CHOICE.

Of course, unless you're running so many blacklists that the time required to go through them all is excessive, the more interesting question is "how many spams didn't get caught by my blacklists, and what additional blacklists would catch those without contributing excessively to my false-positive rate." Again, something that looks up "passed" IP addresses in a set of blacklists you're considering and records the results would be instructive.

share|improve this answer
    
Yeah, I was thinking about doing something like that. I'm surprised that someone hasn't already written it-- it's only a few lines of code. –  user10501 Jul 5 '09 at 21:28
1  
It's common that such simple scripts don't get released, as the effort of packaging, documenting, maintaining, releasing, etc exceeds the effort of just rewriting the sucker. Add on top of that the need to rewrite the log parsing part of the program for each MTA, and voila! –  womble Jul 6 '09 at 9:13

In addition to DNSBL you may wish to consider is GeoIP blocking. This is when you take a look at the IP address the connection is coming from, determine the originating country, and then block the connection.

We find that most of our spam comes from countries that we would NEVER expect to receive an email from (China, Brazil, Singapore, etc.)

By blocking these countries completely we have reduced our spam load a HUGE amount before any other sort of more CPU intensive checking is made.

I would estimate that at least 90% (probably more) of all of our spam is caught by GeoIP and less than 10% is actually processed against DNSBL, spamassassin, whitelist, etc.

We used MaxMind's free GeoIP database but I know there are others out there.

To answer your question directly though about DNSBL, we use the following:

sbl-xbl.spamhaus.org bl.spamcop.net dnsbl.cyberlogic.net dnsbl.ahbl.org

We haven't had to update/change from the above in quite some time. I would say that (for us) spamhaus.org tends to catch most stuff that GeoIP doesn't.

Your mileage may vary.

share|improve this answer

If you have spamassassin apply the blocklists, which it (effectivly) does in paralell you can use its logs to do those hit rate calculations.

Once you know what blocklists to trust you can reconfigure your mailservers to block on the most trusted RBL's.

share|improve this answer
    
SpamAssassin is going to be way more CPU-intensive than MTA-level blocking, no? I'm pretty sensitive to that, though I guess I could put up with it for a few days. –  user10501 Jul 5 '09 at 6:43
    
Absolutely, which is why, once you know the safe blocklists you have your MTA filter them. –  LapTop006 Jul 5 '09 at 9:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.