Dovecot MySQL Authentication SlowDown

Question

I have an IMAP/POP3/Webmail server with Dovecot1 2.2.10 with a MySQL backend located on another server. The server runs with default configuration.

...
driver = mysql
default_pass_scheme = plain
...

My problem is that sometimes when it gets overloaded it start refusing authentication (aka it says bad password) of the clients. It does this for all different types of clients (pop3,imap,webmail) so this must be an authentication backend related issue

My first thought is that there aren't enough dovecot authentication processes running.

What can I tune and in what order for this system to be able to handle more client authentication requests simoulteanously?

My MySQL server is not overloaded, it's a quadcore machine with an average 25% CPU usage during the day.

The CPU usage of the mailserver itself spikes up during this period. Memory usage remains normal.

http://i58.tinypic.com/2q9gfpf.jpg

This is a production system so all I can do is adjust some variables which surely won't break it.

Thanks!

Answer 1

Don't know if the same than which happens with Cyrus IMAPd can happen with Dovecot. Let's try.

The Linux kernel gathers real randomness, entropy, from multiple sources and events. This data can be read by any program from /dev/random. Well, sometimes if there's not enough 'real' randomness generated (for example, the server does not contain a hardware accelerated random number generator), this entropy pool can be empty. And while it's empty, any process which waits for that random data from /dev/random, stalls while the entropy pool is starving.

Whenever this stall happens at your Dovecot server, you may check watch -n1 'cat /proc/sys/kernel/random/entropy_avail' and see if the value stays at or around zero. If it does, your Dovecot needs randomness generated faster.

One solution is to install rngd (for example in Ubuntu/Debian it's in rng-tools package) and configure it to use /dev/urandom. rngd showels random data to /dev/random from the sources you configure it to use. Yes, that use semi-randomness in case of /dev/urandom but at least it works.

In Cyrus IMAPd it was the POP3 greeting line with a randomly generated string which consumed /dev/random very fast and with lots of connections lead to a complete stall lasting up to several minutes.

---

Completely another option is that your webmail application does not nicely handle MySQL connections, so max_connect_errors in MySQL gets exceeded and MySQL starts rejecting your Dovecot server. In this case FLUSH HOSTS; at MySQL server MySQL command line should help.

Should this be the root cause for your Dovecot problems, then you should raise MySQL max_connect_errors limit.

I know my reply is very far-fetched and a complete shot in the dark, but you never know!