1

I've just created a Lambda in AWS that requires access to RDS and, as a result, has to be a VPC Lambda function instead of a regular one. As I understand it, this means I basically have no choice but to create an S3 endpoint in order to access S3 resources (it seems a bit wild to me that accessing both S3 and RDS at the same time requires this workaround; it seems like this would be a common use case).

In any case, I was just about to go through with generating the endpoint when this warning message popped up on the web console:

Warning

When you use an endpoint, the source IP addresses from your instances in your affected subnets for accessing the AWS service in the same region will be private IP addresses, not public IP addresses. Existing connections from your affected subnets to the AWS service that use public IP addresses may be dropped. Ensure that you don’t have critical tasks running when you create or modify an endpoint.

I do have public-facing EC2 instances which make heavy use of S3 (uploading files, downloading files, processing files, etc.). It's not clear exactly what they mean by "existing connections...may be dropped". Is this a one time thing or will adding the endpoint mean that I will have to reconfigure how I access the S3 resources?

For example, since I'm not using the gateway, currently all of my instances (almost all of which use boto/boto3 to access S3) are using "Virtual Host" addressing style but the S3 endpoint expects the path addressing style.

Will all of my instances using virtual host addressing style still work? Or do they all need to be adjusted once I create the endpoint? I'd like to avoid disruption as much as possible.

7
  • The "workaround" you speak of is unnecessary if you simply use a NAT Gateway or NAT Instance. S3 is outside your VPC, RDS (done correctly) is inside. The S3 endpoint gives S3 a presence inside your VPC. Where did you see that S3 endpoints require path addressing style? That doesn't really make sense. Jul 31, 2018 at 23:21
  • Using a NAT gateway means that you access resources via public internet and as I understand it you are charged for this. If you set up a S3 endpoint within the vpc, you use path addressing style to access it. That's based on this answer to a question regarding lambda and s3 stackoverflow.com/questions/39779962/… Aug 1, 2018 at 1:42
  • That other answer is interesting, but I don't believe the conclusion there is fully correct -- it fixes things by accident. For the first few minutes after initial bucket creation, there's very likely an issue with global-virtual-style addressing (bucket.s3.amazonaws.com), if you are using an endpoint without also having NAT in place, but after that it should all work quite normally. I almost always use regional-virtual-style (bucket.s3.us-east-2.amazonaws.com) in my code, which may be why -- so I'll have to validate the assertions. I will try to confirm this as soon as possible. Aug 1, 2018 at 10:57
  • 1
    Also, you're correct that NAT incurs additonal cost, but you have to remember the order in which these things came into being. S3, then RDS, then VPCs and NAT Instances, then RDS inside VPC, then Lambda functions (Internet only, no VPC), then S3 Endpoints, then NAT Gateways, then Lambda inside VPC. The services have evolved over time, and the interdependencies make more sense in that light. Aug 1, 2018 at 11:13
  • 1
    It should be a one-time hit. An S3 endpoint works by giving you a target for your route tables that will send traffic bound for all the S3 public IPs for the region to a different place -- without traversing the Internet Gateway. The routing change is what can disrupt the connections. Only transfers in progress or kept-alive connections still active at the moment when you create the endpoint or update route tables would be impacted. Anything more serious and the problem is somewhere else. Removing the route table entries will fix it, no need ro destroy the endpoint. Aug 1, 2018 at 21:00

0

You must log in to answer this question.

Browse other questions tagged .