Setting up Chef Node: Amazon EC2 instance is create but bootstrap fails

Question

I am trying to get going with Chef and the freebie Chef server from Opscode.

I got my Chef workstation configured on a local VM running Ubuntu 12.04LTS. I downloaded the chef-repo from GitHub as per the instructions.

I downloaded the keys from Opscode (myorg.pem as the organizational key and jgodse.pem as the personal key). I also had a keys from Amazon to use their EC2 API (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY which I set as environment variables). I also created a key at Amazon for use with the instance (deploy.pem). All the keys are in ~chef-repo/.chef/ and in ~/.ssh/ , which is probably overkill, but I just want it to work.

I managed to create an EC2 instance using knife ec2, but knife-ec2 couldn't finish the job and bootstrap the chef-client, because it couldn't log into the newly created server.

Here is my session.

railsdev@localchef:~/Chef/chef-repo$ knife ec2 server create -r 'recipe[apt]'
Instance ID: i-fa4c1b99
Flavor: m1.small
Image: ami-xxxxxxxx
Region: us-east-1
Availability Zone: us-east-1b
Security Groups: default
Tags: {"Name"=>"i-fa4c1b99"}
SSH Key: deploy

Waiting for instance.....................
Public DNS Name: ec2-107-21-188-230.compute-1.amazonaws.com
Public IP Address: 107.21.188.230
Private DNS Name: ip-10-10-206-167.ec2.internal
Private IP Address: 10.10.206.167

Waiting for sshd...done
Bootstrapping Chef on ec2-107-21-188-230.compute-1.amazonaws.com
Failed to authenticate root - trying password auth
Enter your password: 
ERROR: Net::SSH::AuthenticationFailed: [email protected]
railsdev@localchef:~/Chef/chef-repo$ 

I was able to ssh into this instance using

jgodse/Chef/chef-repo> ssh -i ./.chef/deploy.pem [email protected]

Any clues as to why this wont authenticate with Knife EC2?

My ~chef-repo/.chef/knife.rb file looks like this:

current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "jgodse"
client_key               "#{current_dir}/jgodse.pem"
validation_client_name   "myorg-validator"
validation_key           "#{current_dir}/myorg-validator.pem"
chef_server_url          "https://api.opscode.com/organizations/myorg"
cache_type               'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{current_dir}/../cookbooks"]

knife[:aws_access_key_id]      = ENV['AWS_ACCESS_KEY_ID']
knife[:aws_secret_access_key]  = ENV['AWS_SECRET_ACCESS_KEY']

# Default flavor of server (m1.small, c1.medium, etc).
knife[:flavor] = "m1.small"

# Default AMI identifier, e.g. ami-12345678
knife[:image] = "ami-xxxxxxxx"

# AWS Region
#knife[:region] = "us-east-1"

# AWS Availability Zone. Must be in the same Region.
knife[:availability_zone] = "us-east-1b"

# A file with EC2 User Data to provision the instance.
#knife[:aws_user_data] = ""

# AWS SSH Keypair.
knife[:aws_ssh_key_id] = "deploy"

Answer 1

Er...you're trying to ssh in as root. EC2 by default does not allow this even with your keyfile login by default:

ERROR: Net::SSH::AuthenticationFailed: root@

While your test command uses ubuntu@.

You can "allow" root access by modifying the authorized_keys file under /root/.ssh (or adding the key yourself).

Answer 2

There were a couple of problems.

The script was trying to use "root", as it is the default user id. To fix this, I had to add one line to the knife.rb file:

knife[:ssh_user]="ubuntu"

The second problem was that ssh didn't know how to find the key (because it was deploy.pem). So I had to do the following:

ubuntu> exec ssh-agent /user/bin/bash    #This causes the terminal to disappear
ubuntu> ssh-add /home/jgodse/.ssh/deploy.pem   

Once that was done, I reissued the command

knife ec2 server create -r 'recipe[apt]'

and I got further.

Answer 3

Try running the command like below:

knife ec2 server create --image ami-xxxx -i /root/.ssh/deploy.pem --flavor t1.micro -x root --groups ChefGroup -Z ap-southeast-1a -r "recipe[apt]"