6

I'm following the docs on fetching certificates from s3 when a new instance is deployed to elastic beanstalk. The instructions are fairly straightforward: create a config file under app-root/.ebextensions that reads:

Resources:
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Auth:
          type: "s3"
          buckets: ["elasticbeanstalk-us-west-2-123456789012"]
          roleName: 
            "Fn::GetOptionSetting": 
              Namespace: "aws:autoscaling:launchconfiguration"
              OptionName: "IamInstanceProfile"
              DefaultValue: "aws-elasticbeanstalk-ec2-role"
files:
  # Private key
  /etc/pki/tls/certs/server.key:
    mode: "000400"
    owner: root
    group: root
    authentication: "S3Auth"
    source: https://s3-us-west-2.amazonaws.com/elasticbeanstalk-us-west-2-123456789012/server.key

where the links point to the location on your certificate / key / whatever.

However, when I deploy my application bundle (zip), the build fails with [Errno 404] HTTP Error 404... suggesting that my files cannot be found at the specified location contrary to the fact that the files are at the precise specified location since I can aws s3 cp s3://<pasted-link-to-file> . them from my command line (same region of course).

Also, on the surface, it doesn't seem to be a permissions problem because I'd expect something like "Access Denied" if it was; the bucket policy grants the following actions to aws-elasticbeanstalk-ec2-role:

"Action": [
    "s3:ListBucket",
    "s3:ListBucketVersions",
    "s3:GetObject",
    "s3:GetObjectVersion"
]

I also attached full s3 access policy to aws-elasticbeanstalk-ec2-role in IAM.

But the fact that I can access the files from cli and aws-elasticbeanstalk-ec2-role cannot find them, suggests that a) something could be wrong with my setup: files on s3, s3 permissions, config in .ebextensions or b) aws docs on this matter are completely out of whack.

  • This is a bit of a broad question but can anyone suggest what else could go wrong with this sort of setup?

2 Answers 2

3

Solved it by explicitly specifying:

roleName: "aws-elasticbeanstalk-ec2-role"

instead of:

roleName: 
  "Fn::GetOptionSetting": 
    Namespace: "aws:autoscaling:launchconfiguration"
    OptionName: "IamInstanceProfile"
    DefaultValue: "aws-elasticbeanstalk-ec2-role"

in .ebextensions/my_conf.config.

This could be because the config file key and the S3 policy refer to "roleName" and not "IamInstanceProfile".

4
  • 2
    Just to add a little color since I recently got bitten by the same issue, the 404 message happens because in order to get the credentials for the role, elastic beanstalk queries an endpoint. When the role is invalid/does not exist, that endpoint returns 404 and that's getting passed along as the error message. Jun 6, 2018 at 6:40
  • 1
    I'm having this very same issue, but the fix does not work. I've verified that the role and buckets name are correct, but it does indeed seem like the 404 originates from creating the auth token rather than from fetching the actual S3 resource (with authentication in place, even fetching any public S3 asset fails with a 404 - but works without auth).
    – JHH
    Aug 24, 2018 at 8:51
  • I also have several other EB's working with identical config files (but different instance profiles/roles, although with equivalent permissions) but this very one fails with the 404. From inside the node application I am able to access my S3 resources, so the instance profile, bucket policies etc are indeed correct, but from the EB deploy script I still get the 404 when trying establish the S3Auth. Does anyone know how I could debug this with more details? If SSH:ing into the machine, which scripts should I run to emulate what's happening during deployment and perhaps add additional logs?
    – JHH
    Aug 24, 2018 at 8:53
  • An interesting fact is that if I deliberately misspell the name of my role, the outcome is the same. So it's like the 404 is related to not being able to find my role.
    – JHH
    Aug 24, 2018 at 8:54
2

After doing some troubleshooting, I realized the issue with :

"Fn::GetOptionSetting": 
  Namespace: "aws:autoscaling:launchconfiguration"
  OptionName: "IamInstanceProfile"
  DefaultValue: "aws-elasticbeanstalk-ec2-role"

This option returns the Instance Profile. However, to fetch credentials, we need the AWS Role associated with the instance profile.

Following command will return 404:

curl -vvv http://169.254.169.254/latest/meta-data/iam/security-credentials/{Instance-Profile}

However, following command works and returns temporary credentials:

curl -vvv http://169.254.169.254/latest/meta-data/iam/security-credentials/{Instance-ROLE}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .