Amazon recently introduced Target Tracking Policies for EC2 Auto Scaling.
In my production service, I am using two separate auto-scaling groups to support hybrid auto-scaling with a mixture of Spot and On-Demand instances. What I want is that my CPU usage should not exceed 70%, and it should use Spot instances whenever possible but fall back to On-Demand instances if necessary.
First, I set both Auto-Scaling groups (Spot and On-Demand) to use Target Tracking for 70% CPU load and set the minimum size of both groups to one. The traffic on my service is quite predictable (no sudden boost, more traffic during the day, few traffic during the night).
At one point, there were two On-Demand and two Spot instances running. The system had just scaled down because the CPU load of the five servers became very low (around 35%). With the four servers, the CPU load went up and after a few minutes briefly crossed the 70% mark (maybe there was a very minor traffic boost at that time).
The system decided conservatively to scale up again, but as both auto-scaling groups made the decision independently at the same time, two instances were started (one Spot and one On-Demand instance). At this point, there were now six servers running. After a while it scaled down again and finally reached a setup of running four instances.
To avoid that effect, I now changed the setup as follows:
- On-Demand: Target 70% CPU usage, one server minimum
- Spot: Target 65% CPU usage, one server minimum
My assumption is that it should help to prevent that scenario I described. I would expect that the On-Demand group scales down earlier than the Spot group (which is desirable, anyway, as they are more costly). And I expect that the Spot instances scale up sooner, which should protect against unnecessary upscaling from the On-Demand group.
That is my expectation, but I did not find much details in the documentation to confirm it. Can someone shed some light on how the new Target Tracking scaling works in details, and how to apply it to a hybrid setup with Spot and On-Demand instances?
- If I set the target to 70% CPU utilization, when will it decide to scale up and when to scale down?
- If I have two Auto-Scaling groups, one with a 70% CPU utilization target and the other with 65%, when will it decide to scale up or down? Will it always prefer to scale down the 70% group? Will it always prefer to scale up the 65% group?
- What happens if the prices in the Spot market suddenly rise to exceed my bid limit. Will the On-Demand auto-scaling group take over?
- Is my understanding correct that manually defining the number of desired instances has only a short-term effect and will be automatically adjusted by the Auto Scaling policy?
- For example, if it scaled down to the minimum during the night and scaled up again next day, does it mean that the initial "number of desired instances" settings from the previous day are now obsolete? In other words, do I need to worry only about setting reasonable value for minimum and maximum, and will AWS will figure out the rest?