5

I am trying to retrieve some files from a private s3 bucket to a filesystem location elastic beanstalk ec2 instance, but with no success.

I've created a bucket named dev-config containing a file named local.properties.

I've created a IAM policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::dev-config",
                "arn:aws:s3:::dev-config/*"
            ]
        }
    ]
}

And associated that policy to a IAM role, that in turn is associated with the EC2 instance. I have confirmed that I can fetch files from the s3 bucket using the aws-cli without providing any additional credentials. i.e. aws s3 ls s3://dev-config/local.properties

To my project I've added the following file:

.ebextensions/01_files.config

"/usr/share/tomcat7/lib/local.properties" :
    mode: "000777"
    owner: ec2-user
    group: ec2-user
    source: http://s3.amazonaws.com/dev-config/local.properties

I've also tried a few variations of the source url

    source: http://dev-config.s3.amazonaws.com/dev-config/local.properties
    source: http://dev-config.s3.amazonaws.com/local.properties
    source: s3://dev-config/local.properties

And I've also tried adding an authentication attribute with no success (there seem to be no docs on possible values for authentication). authentication: S3Access

None of the approaches have worked so far.

In some cases I get access denied messages in the logs:

    <?xml version="1.0" encoding="UTF-8"?>
      <Error><Code>AccessDenied</Code><Message>Access Denied</Message>
      <RequestId>blahblah</RequestId>
      <HostId>blahblah</HostId>
    </Error>

In other cases I have had error messages in the local.properties file itself PermanentRedirect The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. dev-config dev-config.s3.amazonaws.com
blahlblah blahlblah

Has managed to get this working?

2 Answers 2

4

After taking a look at this answer to Using environment properties with files in elastic beanstalk config files I added the following section to the .ebextensions/01_files.config

Resources:
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Access:
          type: S3
          roleName: aws-elasticbeanstalk-ec2-role
          buckets: dev-config

and updated the s3 url to include the bucket name in the host, so the final file looked like this:

"/usr/share/tomcat7/lib/local.properties" :
    mode: "000777"
    owner: ec2-user
    group: ec2-user
    source: https://dev-config.s3.amazonaws.com/local.properties

Resources:
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Access:
          type: S3
          roleName: aws-elasticbeanstalk-ec2-role
          buckets: dev-config

This enabled the elastic beanstalk ec2 instance to use the IAM role associated with it to access the s3 bucket containing the files.

PS: For this configuration to work, make sure that you've granted access to the S3 bucket in question to the aws-elasticbeanstalk-ec2-role principal. You can get the ARN from IAM console.

4
  • This just doesn't work for me. Always a 403. If I assign the permission to the role manually it works. Am I missing something else in the file?
    – Strelok
    Aug 26, 2015 at 6:10
  • Sorry to hear that. I had this working, I'll go back and check the details of the answer against the implementation.
    – diffa
    Aug 28, 2015 at 9:34
  • I think I soved. You had to give aws-elasticbeanstalk-ec2-role access to the s3 bucket! Doh! After you do that this ebextensions config works as this only instructs the deployer to auth with that role.
    – Strelok
    Aug 31, 2015 at 0:15
  • In original question I mentioned that the once I had created the IAM policy, I then "associated that policy to a IAM role, that in turn is associated with the EC2 instance". The answer implies that is aws-elasticbeanstalk-ec2-role, but I didn't specify it. In practice, I would create another role for your app on elastic beanstalk otherwise all of your EB instances would be able to access the bucket, which might not be what you want.
    – diffa
    Aug 31, 2015 at 10:48
1

Try with this IAM. It works for me.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::dev-config",
                "arn:aws:s3:::dev-config/*"
            ]
        }
    ]
}

If you need to have read/write/delete permissions you need something like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": [
                 "arn:aws:s3:::dev-config",
                 "arn:aws:s3:::dev-config/*"
            ]
        }
    ]
}

Regards.

1
  • 2
    s3:* should already cover all of those.
    – ceejayoz
    Mar 23, 2015 at 15:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .