0

Helo, I have empty AWS ECS Cluster but I am unable to put instances into it. I wanted to use Launch templates and Autoscaling Group, but I am unable to assign created EC2 Instance.

The issue is in shown in ecs-agent.log

level=error time=2020-10-17T23:23:37Z msg="Unable to register as a container instance with ECS: RequestError: send request failed\ncaused by: Post \"https://ecs.eu-central-1.amazonaws.com/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" module=client.go
level=error time=2020-10-17T23:23:37Z msg="Error registering: RequestError: send request failed\ncaused by: Post \"https://ecs.eu-central-1.amazonaws.com/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" module=agent.go

Notes:

  • Using AMI ami-0eff571a24849e852
  • Cluster name: debug
  • Region is eu-central-1
  • Instance has no public IP
  • Instance is in 10.10.100.0/24 subnet (10.10.100.14) and VPN subnet is 10.10.0.0/16
  • Instance can reach the internet through NAT Instance:
[ec2-user@ip-10-10-100-14 ecs]$ ping google.com
PING google.com (216.58.212.142) 56(84) bytes of data.
64 bytes from ams15s21-in-f14.1e100.net (216.58.212.142): icmp_seq=1 ttl=109 time=50.1 ms
64 bytes from ams15s21-in-f142.1e100.net (216.58.212.142): icmp_seq=2 ttl=109 time=40.1 ms
  • DNS to outside is resolving fine
[ec2-user@ip-10-10-100-14 ecs]$ nslookup google.com
Server:     10.10.0.2
Address:    10.10.0.2#53

Non-authoritative answer:
Name:   google.com
Address: 216.58.212.142
  • Just to be sure, I have created Endpoints from VPC and Subnet where Instance is to ECS
  • I have attached the security group with no restrictions for test
  • ecs.config:
ECS_CLUSTER=debug
ECS_BACKEND_HOST=
[ec2-user@ip-10-10-100-14 ecs]$ nslookup ecs.eu-central-1.amazonaws.com
Server:     10.10.0.2
Address:    10.10.0.2#53

Non-authoritative answer:
Name:   ecs.eu-central-1.amazonaws.com
Address: 10.10.100.219
  • Security group where the instance has one rule and which allows ALL traffic from sg where NAT instance is. (Because I cannot upload images here yet, I have to use another provider) https://ibb.co/kh7tb2J

Does anyone have any suggestions?

  • How does the nACL this subnet uses look like? – Mugurel Oct 18 at 0:55
  • @Mugurel Hi, it seems like this is the cause. After I edit the security group where instance sits to allow all incoming traffic - it works! So I wonder how should the security group look like if I am using NAT instance with a separate security group? (I will update question with sg scheme) – Filip Niko Oct 18 at 1:02
0

I have solved this issue.

The problem was caused by VPC Endpoints. I placed them in the wrong subnet and security group.

I have moved them to the public subnet (next to NAT Instance) and in the same group as NAT Instance (allow incoming traffic from sg where my EC2 is).

According to docs:

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/vpc-endpoints.html

The security group attached to the VPC endpoint must allow incoming connections on port 443 from the private subnet of the VPC.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.