3

I am trying to deploy a rails application on Amazon's Elastic Beanstalk. I am not very well-versed in network/server environments, so I picked up some knowledge along the way by reading documentations and researching whatever technologies are present in my stack, so bear with me if I make any mistakes.

So here's what I am trying to achieve:

I want to deploy my Rails application with Docker on Elastic Beanstalk with my instances launched in a VPC with two public facing subnets and an associated RDS instance. So far so good, I got the stack set up, but upon deployment I get an error which indicates that my EC2 instance is not able to connect to the Postgres RDS instance:

[91mrake aborted! [0m[91mPG::ConnectionBad: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

So, in essence my feeling is that I got quite far with the exception being that my RDS instance cannot be reached by my EC2 instance. My guess is that there are non-satisfiable Security Groups even though I tried to make sure to add both the EC2 instance and the RDS instance into the same security group where incoming traffic on port 5432 (Postgres) is allowed by the Security Group itself.

I added a Security Group with my Ip's CIDR and could confirm that the database is reachable from my machine. That means that the connection parameters I specified in my environment are correct and that it's not related to wrong credentials/configuration.

What could be the culprit for my EC2 instance not being able to establish a connection to my RDS instance ? Again, I'm almost positive on the fact that it lies within my Security Group settings, but I haven't been able to narrow down the source of the problem.

If you require any more information (which I intentionally left out for brevity) please let me know (for example security groups for VPC, EC2 or RDS, etc.).

1

Since RDS requires you to have two availability zones when deploying in a VPC, you need to make sure that beanstalk is able to get to both of them via Network ACLs as well as the permissions for the instance based security groups.

Only your ELB and your NAT instance/NAT gateway need to be public subnets, everything else should be in private subnets.

Security groups are stateful and network groups are stateless so while you only need to allow inbound rules for the security groups, you need to allow BOTH inbound and outbound ports from your beanstalk subnet to both RDS subnets using Network ACLs. See Security in Your VPC.

Here is a sample eb create to create the beanstalk environment (replace square bracketed strings):

eb create [BEANSTALK_ENVIRONMENT] --instance_type m3.medium --branch_default --cname [BEANSTALK_CNAME] --database --database.engine postgres --database.version [x] --database.size 100 --database.instance db.m4.large --database.password xxxxxxxxx --database.username ebroot --instance_profile [BEANSTALK_EC2_IAM_PROFILE] --keyname [SSH_KEY_NAME] --platform "64bit Amazon Linux 2015.03 v1.3.0 running Ruby 2.2" --region us-east-1 --tags tag1=value1,tag2=value2 --tier webserver --verbose --sample --vpc.id [vpc-xxxxxx] --vpc.dbsubnets [subnet-db000001,subnet-db000002] --vpc.ec2subnets [subnet-ec200001] --vpc.elbsubnets [subnet-elb00001] --vpc.elbpublic --vpc.securitygroups [sg-00000001] --sample --timeout 3600

subnet-db000001 NETWORK ACL RULES:

Inbound: Port Range: 5432, Source [subnet-ec200001 (as ip range)], Allow
Outbound: Port Range: 5432, Source [subnet-ec200001 (as ip range)], Allow

subnet-db000002 NETWORK ACL RULES:

Inbound: Port Range: 5432, Source [subnet-ec200001 (as ip range)], Allow
Outbound: Port Range: 5432, Source [subnet-ec200001 (as ip range)], Allow

subnet-ec200001 NETWORK ACL RULES:

Inbound: Port Range: 5432, Source [subnet-db000001 (as ip range)], Allow
Inbound: Port Range: 5432, Source [subnet-db000002 (as ip range)], Allow
Outbound: Port Range: 5432, Source [subnet-db000001 (as ip range)], Allow
Outbound: Port Range: 5432, Source [subnet-db000002 (as ip range)], Allow

subnet-elb00001 NETWORK ACL RULES:

Inbound: Port Range: 80, Source 0.0.0.0/0, Allow
Inbound: Port Range: 443, Source 0.0.0.0/0, Allow
Outbound: Port Range: 80, Source 0.0.0.0/0, Allow
Outbound: Port Range: 443, Source 0.0.0.0/0, Allow

An additional note about Network ACLs -- many services don't respond on the original port but use an ephemeral port. So you may have to add the following to the inbound AND outbound network ACLs for subnets with EC2 instances:

Outbound: Port Range: 1024-65535, Source 0.0.0.0/0, Allow
Outbound: Port Range: 1024-65535, Source 0.0.0.0/0, Allow

There are also several useful scenarios in Recommended Network ACL Rules for Your VPC.

I hope this helps.

  • I'll try that and get back to you! Thanks in advance. Would you be ready to answer any questions that might come up ? – the_critic Mar 8 '16 at 16:36
  • sure. one other note about Network ACLs. Many services don't respond on the original port but use an ephemeral port. So you may have to add the following to the inbound AND outbound network ACLs so your beanstalk instance can get to the internet for gem installs, etc: Custom TCP Rule, TCP(6), 1024-65535, 0.0.0.0/0, ALLOW. See docs.aws.amazon.com/AmazonVPC/latest/UserGuide/… – Peter M Mar 8 '16 at 16:46
  • This is the correct chat room if you'd like to join: chat.stackexchange.com/rooms/36705/… – the_critic Mar 8 '16 at 17:28
  • I'd like to thank you for your answer, however I can't get it to work. If I set it up like you say I get issues with my load balancer saying that the underlying EC2 instances are "out of service" (due to health checks failing). Is there any way I can debug the whole stack ? (ELB -> VPC -> EC2). I'm aware of the fact that the health checks test port 80 on my EC2 instance, but for some reason I can't make the Load Balancer work (possibly due to a misconfiguration on my part). I deployed the sample app on EB, so it should be able to deploy (which it does, but then fails after status "OK" by ELB) – the_critic Mar 8 '16 at 22:30
  • If the app deploys successfully the load balancer should be able to pass the health check, which it does not. A connection timeout follows and EB enters "Severe" status. – the_critic Mar 8 '16 at 22:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.