Chef Server on Amazon 2014.03 fails to configure

Question

I'm trying to configure Chef Server 11.1.8 on Amazon Linux 2014.03 (on EC2 instance).

The RPM installs OK:

sudo rpm -Uhv private-chef-11.1.8-1.el6.x86_64.rpm

then run

sudo private-chef-ctl reconfigure

The reconfigure hangs while installing RabbitMQ (see below).

There is a discussion on this here but the file in question has changed and the solution isn't applicable any more.

Any ideas?

- configure service runit_service[rabbitmq]

Recipe: private-chef::rabbitmq execute[/opt/opscode/bin/private-chef-ctl start rabbitmq] action run ================================================================================ Error executing action run on resource

execute[/opt/opscode/bin/private-chef-ctl start rabbitmq]

Mixlib::ShellOut::ShellCommandFailed ------------------------------------ Expected process to exit with [0], but received '1' ---- Begin output of /opt/opscode/bin/private-chef-ctl start rabbitmq ---- STDOUT: warning: rabbitmq: unable to open supervise/ok: file does not exist STDERR: ---- End output of /opt/opscode/bin/private-chef-ctl start rabbitmq ---- Ran /opt/opscode/bin/private-chef-ctl start rabbitmq returned 1

Resource Declaration:

In /opt/opscode/embedded/cookbooks/private-chef/recipes/rabbitmq.rb

79: execute "#{opc_ctl} start rabbitmq" do 80: retries 20 81: end 82:

Compiled Resource:

Declared in /opt/opscode/embedded/cookbooks/private-chef/recipes/rabbitmq.rb:79:in

`from_file'

execute("/opt/opscode/bin/private-chef-ctl start rabbitmq") do
action "run" retries 0 retry_delay 2 command "/opt/opscode/bin/private-chef-ctl start rabbitmq" backup 5
returns 0 cookbook_name :"private-chef" recipe_name "rabbitmq" end

Running handlers: [2014-07-10T06:51:52+00:00] ERROR: Running exception handlers Running handlers complete

[2014-07-10T06:51:52+00:00] ERROR: Exception handlers complete [2014-07-10T06:51:52+00:00] FATAL: Stacktrace dumped to /opt/opscode/embedded/cookbooks/cache/chef-stacktrace.out Chef Client failed. 2 resources updated in 44.887268777 seconds [2014-07-10T06:51:52+00:00] ERROR: execute[/opt/opscode/bin/private-chef-ctl start rabbitmq] (private-chef::rabbitmq line 79) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1' ---- Begin output of /opt/opscode/bin/private-chef-ctl start rabbitmq ---- STDOUT: warning: rabbitmq: unable to open supervise/ok: file does not exist STDERR: ---- End output of /opt/opscode/bin/private-chef-ctl start rabbitmq ---- Ran /opt/opscode/bin/private-chef-ctl start rabbitmq returned 1 [2014-07-10T06:51:52+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Answer 1

Run a private-chef-ctl rabbitmq restart and then try a reconfigure again.

And I'll state here that I'm an engineer with Chef, in the interest of full disclosure.

There is a known issue with Enterprise Chef (which I see you're installing) where rabbitmq will hang like this. The issue in question is that we have a script that detects when rabbitmq starts and holds the reconfigure while waiting for rabbitmq to fully start. There is an issue (perhaps a race condition, we're not 100% sure) where this script just hangs. It doesn't fail in this way predictably however, although when it does fail it seems to get stuck this way on every reconfigure. We haven't yet determined the exact cause of this, but we're investigating and hope to have it fixed for the next Chef server release. Restarting rabbit and trying again gets around this issue.