How are autoscaling group termination policies combined?

Question

When creating an autoscaling group I can choose an ordered list of termination policies for its instances. Amazon's documentation states that

You can use these policies individually, or combine them into a list of policies that Auto Scaling uses when terminating instances.

But it glosses over the specifics of how these policies are combined and when the "fall through" happens to the next policy in the list, i.e. under what conditions each policy fails and moves on to the next policy in the list.

For example, I have a policy list ["OldestInstance", "Default"] in my group and yet after scaling up and then down, the scaling group proceeded to terminate by newest (and healthy) instance (newer by a large margin), and I can't figure out why.

Additionally, according to the same doc, default policy is actually itself a combination of policies, and includes OldestLaunchConfiguration and ClosestToNextInstanceHour as two of its steps. If I have a list that includes ["OldestLaunchConfiguration", "ClosestToNextInstanceHour", "Default"], does it evaluate OldestLaunchConfiguration and ClosestToNextInstanceHour twice?

Lastly, does the termination consider load balancer? For example, if my new instance failed to initialise properly and is not in-service with the load balancer, and OldestInstance is in effect, will scale-down action kill the unhealthy instance first even though it's newer?

Answer 1

Splitting this up into the individual questions: 1) When there are multiple termination policies, how are they applied - It will try to do the first policy first, and if there is a tie move on to the next one. If it goes through all the policies you've selected and there is still a tie, it will pick one of the instances that tied at random

2) How could it have terminated the newest instance when the first termination policy was 'oldest instance' - From the termination policy document, the AutoScaling Group (ASG) will always try and balance the AZ's first, and then apply your termination policy within the AZ with the most instances. So if you have 2 AZ's, first with 2 instances and the second with 1 instance, the ASG will terminate the oldest instance in the first AZ. - This wouldn't have applied at the time your asked the question, but if you're using an ASG with the new option to mix instance types and purchase options, it will try and keep your spot/on demand balance before taking your termination policy into account as well. https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-instance-termination.html

3) Does the termination consider the health status of the Elastic Load Balancer (ELB)? - No, healthcheck replacements are separate from scaling. If an instance is marked as unhealthy by the load balancer, the ASG will replace it IF: A) the ASG healthcheck type is set to ELB B) The instance has been in service longer than the ASG's Health Check Grace Period C) An instance in this ASG has ever been marked healthy by that ELB/Target Group - https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-add-elb-healthcheck.html - https://docs.aws.amazon.com/autoscaling/ec2/userguide/attach-load-balancer-asg.html