How to identify how our qmail queue is so big and how to deal with it

Question

Some how our server is being over run by outgoing emails, which we think are spam. We normally use plesk to administer it but it is currently crashing.

If we do a

ps aux | grep 'qmail'

We get about 200 of these

qmailq   25340  0.0  0.0   2640   408 ?        D    11:18   0:00 bin/qmail-queue

How do we identify why this mail queue is so big? Im guessing it could be a compromised PHP script on one of the domains or something like that.

Can someone give us an indication of where to start looking to get to the bottom of this. I realise this is a massively open ended question but any direction would be greatly appreciated.

Answer 1

Someone is connecting, but how? Run netstat -tp.

This might show if someone is forwarding mail directly through a misconfigured mailserver acting as an open relay (port 25), or if they're pushing stuff through your scripts (port 80/443).

If you find one traffic source and can't get the underlying fix made quickly, getting a block in ASAP, either via application configuration or iptables. Keep it a priority though, because whatever one spammer finds, another one will in short order.

Have you looked at the mail content directly? It should be in the filesystem somewhere in /var/spool.

Answer 2

Start with your qmail logs:

tail -f /var/log/qmail/current | tai64nlocal

Watch to see who's sending messages and where they're being sent. If it seems most messages are coming from SMTP, look at the SMTP logs:

tail -f /var/log/qmail/smtpd/current | tai64nlocal

Look at the IP addresses of message senders to see if they are coming from a single source. If you can narrow down the sender addresses, you can create a rule in /etc/tcp.smtp to block that address, or RBL it. Are you using rblsmtp as part of your qmail-smtp pipeline?

If the messages aren't coming from SMTP or they're coming from the localhost because your web pages are posting through SMTP, take a look at your web server logs to see if you can ascertain which IP address(es) are sending the messages.

Answer 3

/var/qmail/bin/qmail-qstat

or better

/var/qmail/bin/qmail-qread

will be the starting point.

Then locate a SPAM message ID and search for a file with that name in /var/qmail/queue (find /var/qmail/queue -iname ).

Find common characteristic to all SPAM (say IP/Subject, From, etc.) and use qmHandle to remove emails from queue and take steps to make sure that SPAM stop coming.