I am using an infrastructure platform consisting of docker containers running on a CentOS host.
- Scaling multiple containers through docker compose.
- Consul for service discovery.
- Registrator for service registry
- HAProxy as a load balancer
- Consul-template for generating haproxy configuration each time the container is added/removed.
Each time we update our build, new containers are spawned and old containers are removed. The problem is containers are removed instantaneously without regards to the existing requests being served by the containers. Ideally, I should have a signal to mark the containers in maintenance or drain mode so that existing connections can be served and then gracefully removed from haproxy first and then physically.
Has anyone faced this problem before or done the similar autoscaling workflow through containers.