Create an Amazon EC2 instance with automatic access to private S3

Question

I was wondering. Say I have a file in a private S3 bucket, and I then deploy an EC2 instance which has something like that in its rc.local:

GET http://privateamazons3url.com

How can I make it so that my EC2 instance that I deploy automatically has access to my private S3 bucket, without me having to explicitly add it manually via the AWS control panel.

The reason I ask is because I am planning to to deploy my EC2 instance remotely using an python boto script from another server using a snapshot image, and the snapshot image will include downloading a file from my private S3 bucket.

Answer 1

Option 1

If your new instance is going to download the file fairly soon after being started, then you can create a time-limited public file:

1. Generate a time-limited URL to the file you want to download. See http://docs.aws.amazon.com/AmazonS3/latest/dev/S3_QSAuth.html
2. Launch your new instance, passing the generated URL in the user data.
3. Inside your new instance, read the user data and download the file.

By doing the above, no credentials are needed on your EC2 instance. However, your public URL is public for a brief time.

Option 2

Launch your new instance in an EC2 "role". See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UsingIAM.html#UsingIAMrolesWithAmazonEC2Instances

When you use an EC2 Role, your instance has auto-generating, auto-expiring access keys provided to it on a regular basis. So you don't have to have credentials kept on your server. Instead, your AWS client will continually request from AWS new keys to use which will change over time.

This way, you can give your EC2 role an IAM policy that allows S3 access to your file(s).

Answer 2

If you want to expose your s3 api info in a query string on the snapshot you can use the following

http://docs.aws.amazon.com/AmazonS3/latest/dev/S3_QSAuth.html

You could also use IAM to make a user that has restricted access to just that bucket also and it will generate a new api key pair to use in the query string.