I'm using chef's knife to launch new ec2 instances. This works generally well, but every 2-3 attempts it fails and I get the following stack:

[Sun, 10 Apr 2011 08:11:23 +0000] INFO: Client key /etc/chef/client.pem is not present - registering
[Sun, 10 Apr 2011 08:11:29 +0000] WARN: HTTP Request Returned 401 Unauthorized: Failed to authenticate as i-XXXXXXX. Ensure that your node_name and client key are correct.
/usr/lib/ruby/1.8/net/http.rb:2101:in `error!': 401 "Unauthorized" (Net::HTTPServerException)
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/rest.rb:234:in `api_request'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/rest.rb:285:in `retriable_rest_request'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/rest.rb:215:in `api_request'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/rest.rb:111:in `get_rest'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/node.rb:525:in `load'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/node.rb:510:in `find_or_create'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/client.rb:218:in `build_node'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/client.rb:145:in `run'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/application/client.rb:222:in `run_application'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/application/client.rb:212:in `loop'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/application/client.rb:212:in `run_application'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/application.rb:62:in `run'
    from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/chef-client:26
    from /usr/bin/chef-client:19:in `load'
    from /usr/bin/chef-client:19

This is a non-deterministic failure (might it have something to do with a race condition?).


Is this still happening for you? The error occured when attempting to use the validation client to automatically generate the new Chef Client. If this occurs, you can check the /etc/chef/client.rb config file for the validation client name and verify it is correct with your Chef Server configuration. If this happens intermittently, you can try rerunning chef-client on the target node with -l debug to get more output about the HTTP request that makes the new client.

| improve this answer | |

While this might not be your exact problem, you need to be aware that creating an instance on EC2 doesn't always succeed. Sometimes, there isn't enough capacity in the AZ you're trying to create your instance in, for that particular class.

Ylastic's Twitter account, which shows informative EC2 error messages which don't always seem to show up on the Amazon service health dashboard, sometimes has messages like this:

#AWS #EC2 (California) : Insufficient Instance Capacity for m1.large

While you're seeing this problem, try creating (and then destroying, you don't want to pay for them forever!) some instances in another region or AZ.

| improve this answer | |
  • The failure I'm experiencing happens right after chef is installed on the remote instance, so it's probably not an issue of AWS resource allocation.It's good to know about that issue too though, thanks. – Ophir Radnitz Apr 13 '11 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.