We're having some major issues with our mail server, normally after a reboot (but not always) where disk io will shoot up from an average 50-100mbps, to 200-300 mbps.
I've narrowed it down to dovecot, killing dovecot brings disk usage to normal immediately. I would say that maybe it's just dovecot processing a backlog after an outage, as we did have an outage for a few hours yesterday, but it settled yesterday evening, then ramped back up again today.
Using atop I've found specific imap accounts that are causing the issues. I've contacted the clients, and found that all the highest io clients have been deleting large quantities of email from their inboxes.
Obviously can't stop them from doing that, but the increased disk io causes a lot of io wait and services stop responding, so we need a way to reduce dovecot's io footprint. I know you can nice something to reduce it's cpu priority, is there something similar for disk priority?
Our server management team says it's as good as it's going to get, and we need to allocate more resources to dovecot, but I'd prefer to rule out the possibility of bad configuration before we throw another server at it, only to have the issues follow us. 300mbps from a few people deleting their email sounds a bit excessive IMHO.
Below is an output of our dovecot configuration from
dovecot -n. Any assistance anyone can provide would be greatly appreciated.
You'll see I've already tried disabling fsync as per this article: http://wiki.dovecot.org/PerformanceTuning. Haven't noticed any considerable improvement. Any suggestions anyone could provide would be greatly appreciated.
Edit: We have one file per user, stored in /var/spool/mail, so I'm guessing it's mbox format.
# 1.0.7: /etc/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login verbose_proctitle: yes mail_privileged_group: mail fsync_disable: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: root