Following rposcibed pattern here: Scenario 3: VPC with Public and Private Subnets and Hardware VPN Access

Summary Description: I have configured a VPC with 1 public Subnet and 2 private Subnets Each subnet is in a different AZ There are two Route Tables: 1 for public Subnet and 1 for 2 private subnets There are two NACLs: 1 for public Subnet and 1 for 2 private subnets There is an internet gateway (routed to the public subnet) There is a Security Group for the EC2 instances There is a Security Group for the RDS instances I have an EC2 instance Apache/Tomcat with web application in the Public Subnet I have an RDS PostgreSQL database deployed in the 2 private subnets (multi-AZ=YES)

This is what I can do:

  1. I can SSH to the EC2 Instance

  2. I can access the Tomcat Manager app on port 80

  3. I can deploy a war file (my application)

  4. If I make the private Subnets public and set the Security Groups and NACLs accordingly, I can access the postgresql database with PGAdmin and I can run the application from my desktop using the RDS instance endpoint as the host. So the database is in good order and can be reached.

Here is what is failing:

  1. When I deploy my application to Tomcat and attempt to access the database it fails: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connecions.

  2. When I ping the RDS endpoint (DNS name) it resolves correctly to the private IP address but cannot reach the node (ALL Traffic is enabled in the NACLs and Security Groups).

So I can get to both EC2 and RDS instances from my desktop from the public internet, but I cannot enable them to connect to each other from within the VPC.

I set all of the NACLs and Security groups to maximum permissiveness just to force a successful connection. But even that is not working.

I will include the pertinent details a little later. If anyone has any suggestions on what to check for would be greatly appreciated!!!

Thanks


I finally figured it out. I'll finish describing the problem and then I will provide the answer. The problem was rooted in the NALCS and Security Groups so I'll share just those details. The routes and internet gateway were straight forward.

VPC Architecture

VPC NACL

In Bound
80  0.0.0.0/0 Allow
Out Bound
All 0.0.0.0/0 Allow

Public Subnet NACL

InBound
80  0.0.0.0/0 Allow
Out Bound
All 0.0.0.0/0 Allow
5432    172.30.4.0/24 Allow (PostgreSQL)

Private Subnet NACL

InBound
5432    172.30.1.0/24 Allow
Out Bound
5432    172.30.1.0/24 Allow 

VPC Security Group

InBound
80  VPC-Security-Group-ID Allow
Out Bound
All 0.0.0.0/0 Allow

Public Subnet Security Group

InBound
80  0.0.0.0/0 Allow
Out Bound
5432    172.30.4.0/0 Allow

Private Subnet Security Group

InBound
5432    172.30.1.0/24 Allow
Out Bound
5432    172.30.1.0/24 Allow 
  • Sounds like something possibly slightly weird in either your route tables, security groups, or NACLs. Can you post them all, in full, please? – Craig Miskell Nov 27 '15 at 8:58

f you know any thing about this (maybe you are a network engineer) you probably see right away what the problem is.
The problem is rooted in these concepts:

  • Ephemeral ports
  • Stateless and Stateful
  • Connection Tracking
  • NACLs are Stateless
  • Security Groups are Stateful

Read about it here:

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Security.html

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_SecurityGroups.html

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_ACLs.html

https://en.wikipedia.org/wiki/Ephemeral_port

VPC Architecture

The corrected confiuration is below. The change that really corrected the connectivity between the Public and Private subnets was enabling response traffic on the Ephemeral ports for both In Bound Public subnet NACL and Out Bound Private subnet NACL rules. I also cleaned up some redundant and insecure Out Bound rules in the NACLS and Security Groups.

VPC NACL

In Bound
80  0.0.0.0/0 Allow
Out Bound
32768-65535 0.0.0.0/0 Allow

Public Subnet NACL

InBound
80  0.0.0.0/0 Allow
32768-65535 172.30.4.0/0 Allow
Out Bound
32768-65535 0.0.0.0/0 Allow
5432    172.30.4.0/24 Allow (PostgreSQL)

Private Subnet NACL

InBound
5432    172.30.1.0/24 Allow
Out Bound
32768-65535 172.30.1.0/24 Allow 

VPC Security Group

InBound
80  VPC-Security-Group-ID Allow
Out Bound

Public Subnet Security Group

InBound
80  0.0.0.0/0 Allow
Out Bound
5432    172.30.4.0/0 Allow

Private Subnet Security Group

InBound
5432    172.30.1.0/24 Allow
Out Bound

This is a working configuration. Hopefully this helps someone. If you see anyhting that can be improved let me know and let me know if you have questions.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.