You indicated in your comment that the instances have no public IP addresses. I'm extrapolating from that comment that your instances likely have no route to the Internet as well.
In order to use ECS, your instances need to have a route to reach (at a minimum) the ECS service endpoints. A route to the Internet can be through an Internet Gateway (IGW), Network Address Translation (NAT), or through an HTTP Proxy. Without a route to reach the ECS service endpoints, the ECS agent will be unable to register itself into your cluster and you will be unable to use those instances with ECS.
The issue for me was that the ECS agent was not starting on the EC2 instance.
And was seeing the below error in the ecs agent logs at /var/log/ecs/ecs-init.log
[INFO] Agent exited with code 5
[ERROR] agent exited with terminal exit code
Following the instructions mentioned here and deleting the json file located at var/lib/ecs/data/ecs_agent_data.json and restarting the ECS agent using the below command is what ultimately worked for me.
/var/log/ecsto see what problems are reported there.