Fast way to filter spam messages from maildir?

Question

I have a Maildir with about 50k emails, about 42k of which are spam (the result of leaving my domain and email up and not using it for 2 years).

I have the following script to try and filter out spam:

#!/bin/sh
for email in "$@"; do
  if ! spamassassin -e < "$email" > /dev/null 2>&1; then
    mv "$email" /tmp/spam
  fi
done

But this is taking forever!

The device I am only able to do this on is limited, it's a NAS with an arm processor...but still!

Is there possibly any faster way to do this? At the moment, workingon a subdir with 1000 files, it's been going for almost 12 hours and found 36 spam messages...

Answer 1

You may start spamd daemon in "local tests only" mode and test messages using spamc command. It should eliminate huge part of spamassassin initialization e.g. parsing perl and configuration files.

Running network tests for huge number old messages (up to two years old) is not cost/time effective IMHO.

AFAIR on my old laptop processing one message in "local only mode" takes spamc around 0.3s (three messages per second or 10_000 messages per hour), processing one message in "network mode" take usually 5-7s over mobile broadband connection. It would be a good idea to run a few network tests in parrallel during mass checks.

You will find a summary of performance related topics in the spamasassin manual.

---

#!/bin/sh
for email in "$@"; do
  # SCORE - "12.7/5.0" on success OR "0/0" on error
  SCORE=`spamc -c < "$email" > /dev/null 2>&1`
  # EXIT - 1 on spam detection, 0 on not-spam or error 
  EXIT=$?
  if [ "$EXIT" = "1" ] ; then
    mv "$email" /tmp/spam
  fi
done