6

I've set up a cluster, container, task and service on ECS.

After I created the service, it appears to be running with '3 desired tasks', however no tasks are being created as pending or running. Effectively it seems to be doing nothing.

My task definition (excluding nulls and empty arrays) is

{
  "containerDefinitions": [
    {
      "portMappings": [
        {
          "hostPort": 5000,
          "protocol": "tcp",
          "containerPort": 25565
        },
        {
          "hostPort": 5000,
          "protocol": "udp",
          "containerPort": 25565
        }
      ],
      "cpu": 0,
      "memoryReservation": 1024,
      "image": "itzg/bungeecord",
      "essential": true,
      "name": "BungeeCord"
    }
  ],
  "compatibilities": [
    "EC2"
  ],
  "taskDefinitionArn": "arn:aws:ecs:us-west-1:949960343466:task-definition/BungeeCordTask:3",
  "family": "BungeeCordTask",
  "requiresAttributes": [
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.21"
    }
  ],
  "requiresCompatibilities": [
    "EC2"
  ],
  "revision": 3,
  "status": "ACTIVE",
}

My cluster has 3 ECS instances, which are all running and have nothing on them: My cluster

My service is here: My service

I don't have any auto-scaling, load balancing or anything set up. But I do have some really default service discovery.

Anybody have any ideas why no tasks are starting?

Edit:

  • I've had a look at the "Stopped tasks" and it's actually empty.
  • I also SSH'd into each of my ec2 instances, and had a look through /var/log: I found a folder /var/log/ecs with some log files, but there are no errors or anything relating to my task specifically. It seems to be successfully polling a amazon web endpoint for updates.
  • I also ran curl google.com to verify that they have external access.
2
  • What does CloudTrail says?
    – titus
    Aug 5, 2019 at 14:15
  • 1
    Just wondering if you found a solution to this problem, I've facing the same issue. Feb 10, 2020 at 9:27

4 Answers 4

2

I'd check your Events tab in the ECS service. In some cases it will tell you there isn't enough memory to start the container. It's a bit tricky to find without knowing, since it's buried in a tab within the service. I'd expect some events must have occured.

0

It could be that the instances are not registered with ECS. Make sure to create instances only by using CloudFormation. Update the services only after all instances are in running state. If you are updating CloudFormation using the command line, make sure that you put in a sleep and allow your EC2 instances to startup first and then update the ECS service.

0

In the Clusters tab, pick the cluster. In the Service open the Tasks tab and switch the view to “Stopped tasks”.

You should see a list of tasks that failed to start and a reason why they failed.

Also check /var/log/amazon/{something about ecr}/* for other possible errors.

It could be that the instances can’t reach the docker registry (network config issue) or that they can’t login if it’s ECR (IAM Role) or needs username / password.

Hope that helps :)

1
  • Hi, thanks for your response :) Please check my edits to the question.
    – Jack3311
    Aug 5, 2019 at 7:27
0

An answer that may help some. Similarly for me, the service was not attempting to start any tasks - no stopped tasks, no events, nothing showing up in CloudTrail.

It turned out that one of the subnets in the service definition no longer existed. The service silently failed to start any tasks with no evidence of the attempts being logged. So, if in doubt, try recreating the service to help ensure all the resources you're trying to use still exist.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.