4

I have Rabbitmq server version 3.5.7-1ubuntu0.16.04.2 installed on Ubuntu 16.04 server. When the server is rebooted Rabbitmq randomly fails to start. Rabbit server is installed from official Ubuntu repository. There is no clustering, it is just one independent server.

When the problem happens this is what gets logged to Journalctl on reboot:

Jun 03 09:50:52 hostname systemd[1]: Starting RabbitMQ Messaging Server...
...
Jun 03 09:51:23 hostname rabbitmq[1067]: Waiting for 'rabbit@hostname' ...
Jun 03 09:51:24 hostname rabbitmq[1067]: pid is 2455 ... 

At 09:51:29 application that tries to use Rabbit is unable to connect: [Errno 111] Connection refused

Few seconds later Rabbit is started according to /var/log/rabbitmq/rabbit@hostname.log:

=INFO REPORT==== 3-Jun-2018::09:52:22 ===
Server startup complete; 6 plugins started.
 * rabbitmq_management
 * rabbitmq_web_dispatch
 * webmachine
 * rabbitmq_management_agent
 * mochiweb
 * amqp_client

But at the same moment, Rabbit goes into error state because of Start-post operation timing out:

Jun 03 09:52:22 hostname systemd[1]: rabbitmq-server.service: Start-post operation timed out. Stopping.
Jun 03 09:52:22 hostname systemd[1]: Failed to start RabbitMQ Messaging Server.
Jun 03 09:52:22 hostname systemd[1]: rabbitmq-server.service: Unit entered failed state.
Jun 03 09:52:22 hostname systemd[1]: rabbitmq-server.service: Failed with result 'timeout'.

Logs in /var/log/rabbit don't show any errors warnings.

Looking at the Systemd unit file I see that Startpost operation executes script /usr/lib/rabbitmq/bin/rabbitmq-server-wait which contains following code:

. `dirname $0`/rabbitmq-env
/usr/lib/rabbitmq/bin/rabbitmqctl wait $RABBITMQ_PID_FILE

After this running command service rabbitmq-server start starts the service without problems.

I am unable to understand what is purpose of this Start-post operation and why it is timing out. What could be causing the problem and how should I proceed researching it?

1

The RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.


You are using an out-of-date and unsupported version of RabbitMQ. Since then there have been improvements with systemd integration. You should use the latest version of RabbitMQ with Erlang 19.3 or later.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.