How should secret files be pushed to an EC2 (on AWS) Ruby on Rails application?

Question

How should secret files be pushed to an EC2 Ruby on Rails application using amazon web services with their elastic beanstalk?

I add the files to a git repository, and I push to github, but I want to keep my secret files out of the git repository. I'm deploying to aws using:

git aws.push

The following files are in the .gitignore:

/config/database.yml
/config/initializers/omniauth.rb
/config/initializers/secret_token.rb

Following this link I attempted to add an S3 file to my deployment: http://docs.amazonwebservices.com/elasticbeanstalk/latest/dg/customize-containers.html

Quoting from that link:

Example Snippet

The following example downloads a zip file from an Amazon S3 bucket and unpacks it into /etc/myapp:

sources:  
    /etc/myapp: http://s3.amazonaws.com/mybucket/myobject 

Following those directions I uploaded a file to an S3 bucket and added the following to a private.config file in the .elasticbeanstalk .ebextensions directory:

sources:
  /var/app/current/: https://s3.amazonaws.com/mybucket/config.tar.gz

That config.tar.gz file will extract to:

/config/database.yml
/config/initializers/omniauth.rb
/config/initializers/secret_token.rb

However, when the application is deployed the config.tar.gz file on the S3 host is never copied or extracted. I still receive errors that the database.yml couldn't be located and the EC2 log has no record of the config file, here is the error message:

Error message:
  No such file or directory - /var/app/current/config/database.yml
Exception class:
  Errno::ENOENT
Application root:
  /var/app/current

Answer 1

Smells like a typo.

The instructions you linked to say, in relevant part:

Customizing your AWS Elastic Beanstalk environment when you deploy your application requires two steps:

1. Create a configuration file with the extension .config and place it in an .ebextensions top-level directory of your source bundle. You can have multiple configuration files in your .ebextensions directory. These files are executed in alphabetical order. For example, .ebextensions/01run.config is executed before .ebextensions/02do.config.

However, you said you placed the .config file in a .elasticbeanstalk directory. Try fixing the directory name.

Answer 2

It is possible (and easy) to store sensitive files in S3 and copy them to your Beanstalk instances automatically.

When you create a Beanstalk application, an S3 bucket is automatically created. This bucket is used to store app versions, logs, metadata, etc.

The default aws-elasticbeanstalk-ec2-role that is assigned to your Beanstalk environment has read access to this bucket.

So all you need to do is put your sensitive files in that bucket (either at the root of the bucket or in any directory structure you desire), and create a .ebextension config file to copy them over to your EC2 instances.

Here is an example:

# .ebextensions/sensitive_files.config

Resources:
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Auth:
          type: "s3"
          buckets: ["elasticbeanstalk-us-east-1-XXX"] # Replace with your bucket name
          roleName: 
            "Fn::GetOptionSetting": 
              Namespace: "aws:autoscaling:launchconfiguration"
              OptionName: "IamInstanceProfile"
              DefaultValue: "aws-elasticbeanstalk-ec2-role" # This is the default role created for you when creating a new Beanstalk environment. Change it if you are using a custom role

files:
  /etc/pki/tls/certs/server.key: # This is where the file will be copied on the EC2 instances
    mode: "000400" # Apply restrictive permissions to the file
    owner: root # Or nodejs, or whatever suits your needs
    group: root # Or nodejs, or whatever suits your needs
    authentication: "S3Auth"
    source: https://s3-us-west-2.amazonaws.com/elasticbeanstalk-us-east-1-XXX/server.key # URL to the file in S3

This is documented here: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/https-storingprivatekeys.html

Answer 3

You can run a bash script and have all the files you need downloaded from S3 using a command line utility such as s3cmd.

I have written a series of articles that cover the customization of AWS Elastic Beanstalk.

For the details about downloading files from S3 you can refer to http://www.hudku.com/blog/security-credentials-setup-customizing/#aws-credentials-setup.sh

Answer 4

Try setting your sources' source to /var/app/ondeck. IIRC, the script runs there and once everything is done, the folder is renamed to /var/app/current so you're probably unzipping to the build that's about to get replaced.