I noticed the following blurb in the batbook:
The best way to flush a full queue is with a command line something like this:
/usr/sbin/sendmail -OQueueSortOrder=filename -q10m -d99.100
/usr/sbin/sendmail -OQueueSortOrder=random -q10m -d99.100← V8.12 and later
/usr/sbin/sendmail -OQueueSortOrder=none -q10m -d99.100← V8.13 and later
Here, the -d99.100 tells sendmail to run in the foreground (so that you can kill it easily when done). The -q10m causes a queue-processing daemon to be launched once each 10 minutes (just like before). You need this because one daemon can seem to hang when delivering mail to a slow host. By running parallel daemons, you avoid this pitfall.
Later on in the same chapter, it talks about setting the interval to be a bad idea:
An hour later, service is restored. First, the default:
/usr/sbin/sendmail -q10mcauses a forked copy of sendmail to start processing the queue. This time, however, the processing is not swift. When a queue fills to 30,000 or more messages, the amount of time it takes to preread the queue (to open and read every message) increases to more than 20 minutes. And those 20 minutes are only for the preread. During those 20 minutes no mail will be sent.
After that, things get worse. Ten minutes later a second sendmail daemon is forked, and it, too, starts to preread the queue. Now, instead of one sendmail daemon opening and reading all messages in a queue, we have two sendmail daemons doing the same thing in parallel.
Contrary to what you might think, twice as much I/O on a disk is not twice as fast. Disks are finite devices that perform a limited number of disk-head moves per second and can transmit only a fixed number of bytes per second. Because the two sendmail daemons are 10 minutes out of step with each other, each is reading and processing separate files. Depending on the size of your in-memory disk cache, neither will likely be able to take advantage of the efficiencies of such caching. In short, two sendmail daemons processing a deep queue in parallel is worse than a single sendmail daemon processing that same queue alone.
And if that weren’t enough, another 10 minutes later a third sendmail daemon starts to process the queue.
By now, the first sendmail daemon might have finished its preread of the queue and might have actually begun to send messages. But even if it has, three sendmail daemons are now processing that single deep queue and a curious thing happens. Because the disk that holds the queue is finite, the addition of a third sendmail daemon slows the operation of the first two. The second one, instead of taking 20 minutes to preread the queue, will now take 30 minutes.
This means that every 10 minutes another sendmail queue-processing daemon is added to the mix. As each is added, each slows all the others that are already running, and it isn’t long before the load on the machine starts to climb and the rate at which messages are delivered falls at an alarming rate. In fact, when this sort of behavior hits a very large-volume site, a sendmail queue-processing daemon can start and seem to never finish.
So I feel like I'm reading two completely different things in the documentation:
- "hey! If your queue is massive, set the queue runner to launch every 10 minutes!"
- "Hey! If your queue is massive, whatever you do, don't set the queue runner to be too short or it will slow stuff down."
What's the appropriate action to take for large queues?