We have an Autoscaling Group in AWS that we manually scale up and down at the time being. Our ASG is currently attached to one Target Group, which has all of our prod servers in it.
Before using the ASG, we would add and remove servers to the Target Groups manually from the Target Group console, and the nodes would deregister after draining properly. We have a very stateful application so the deregistration delay is really important for us.
Yesterday when scaling down through the ASG interface (specifically by asking for a lower number of desired instances), all the connections dropped instantly, which dropped hundreds of connections instantly instead of honoring the five minute draining policy of the target group.
How do I make my ASG honor my draining policy?
I have tried this: AWS ASG With Application LB and Connection Draining but it does not set the connections to "draining", but just "terminating: waiting", and my health checks are healthy so I don't think it is actually stopping new connections/draining.