I need to configure an autoscale group of EC2 instances that on boot will clone a particular branch and commit from a private github repository, then execute code from inside that repository.

I'm using an Ubuntu-derived AMI that has the cloud-init launch system on it, so it seems to me the simplest way to implement this would be a cloud-init user script that installs a GitHub deploy key, uses that to clone the repository, and then starts the appropriate scripts within the cloned repository.

Here's my first attempt (the deploy key, branch and commit hash are inserted by a management process that manages the autoscale cluster before initializing the LaunchConfiguration ):

#!/usr/bin/env python


REPO_URL = 'git@github.com:github_user/MyRepository.git'
BRANCH = 'master'
COMMIT = '6dba9ae2cb77dc30c525ce14aeb82b072c88042b'
USER_HOMEDIR = os.environ['HOME']

    os.makedirs(os.path.join(USER_HOMEDIR, '.ssh'))

# Install the deploy key as the user's default identity file
keypath = os.path.join(USER_HOMEDIR, '.ssh', 'id_rsa')
    with open(keypath, 'w') as keyfile:
    with open(keypath, 'r') as keyfile:
        assert keyfile.read() == DEPLOY_KEY

# openSSH requires identitiy file mode to be 600
os.chmod(keypath, stat.S_IREAD)

# Disable strict host checking for github.com so we don't get the prompt
with open(os.path.join(USER_HOMEDIR, '.ssh', 'config'), 'a') as sshconfig:
    sshconfig.write("\nHost github.com\n\tStrictHostKeyChecking no\n")

# clone the get repo
subprocess.call('git clone %s --branch %s' % (REPO_URL, BRANCH), shell=True)
if COMMIT != 'HEAD':
    os.chdir(os.path.join(WORKING_PATH, 'LFAnalyze'))
    subprocess.call('git checkout ' + COMMIT, shell=True)

This script works from a login shell, but it fails to to run within cloud-init's environment, because the $HOME environment variable isn't set yet (cloud-init runs within an Upstart job at runlevel 2, with access to these environment variables).

git-clone doesn't seem to have an option to pass in SSH options such as would let me use a particular key file. It just looks in ~/.ssh/ for the default identity key. If $HOME isn't defined when this script is executing, how can I tell git-clone where to look for the Deploy Key?

Alternatively, is there some way to make cloud-init execute a script as a particular user, emulating a login shell environment?

Or am I just approaching this entire problem the wrong way? Is there an established best practice or standard way of doing this?

  • try the ssh-agent answer from: stackoverflow.com/questions/4565700/… perhaps ? – Sirex Jun 12 '13 at 19:46
  • 1
    also, look into the $GIT_SSH env var. You can have a script call ssh in the way you wish and have git use that for its ssh connection. – Sirex Jun 12 '13 at 19:48
  • The $GIT_SSH solution worked! I set it to point to a script like this: #!/bin/bash ssh -o StrictHostKeyChecking=no -i /path/to/deploy_key "$@" – leted Jun 13 '13 at 1:10

Normally, I would recommend that you create a working EC2 instance with the SSH key and git repo clone (a working instance with your ideal configuration).

Then create a AMI of that instance and in the launch configuration enter your scripts in the userdata so that the scripts would run each time (basically update git and launch command) when the instance is launched.

Sample script would be (assuming /opt/src is where your repo is):

cd /opt/src
git pull
source run_cmd.sh

This way your script is contained and well maintained. Hope that helps.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.