Using Azure Traffic Manager for an immediate increase in capacity

Question

I have a REST web-service on Azure which has very high but variable load, it's all set-up to auto scale using Paraleap so that it can handle the peak periods but keep costs down when things are quieter.

I have never been able to figure out a way, using any metrics, to predict when a server is going to start maxing out before it actually maxes out! So the solution I have at the minute is a separate programme that constantly checks to see if the server is up, if it starts returning errors then it tells the server to start returning an error message to a certain percentage of users, returning a simple error takes up less of the servers resources which allows the majority of users to still have a service, and then it tells Paraleap to increase the number of instances .. increasing instances takes 10-15 minutes normally, so during this period things aren't great and some users get errors, but ultimately, the new instance kick in and normal service is resumed.

I hoped Azure Traffic Manager would be my solution, my hope was that I could use failover mode, and when a failure was detected on my main web-service, I could divert x% of requests to a backup, which would return the main-service to a working state .. at the same time I would independently tell the main web-service to scale, and when it finished, the traffic manager would divert everything back to the main web-service. In other words, I'd get an instant increase in capacity which would fill the gap whilst I boot up new instances.

Unfortunately, I can't seem to find a way to do this! It looks like Traffic Manager, on detecting a failure, diverts 100% of traffic to the backup. So I'd need to more than double my server capacity just for these moments i.e. have X instance for the main web-service, and x+1 waiting in the backup, a failure with main would diver 100% of requests to backup which would have more capacity, then I would launch more instances for the main, eventually Traffic Manager would send all requests back there, at which point I'd then need to add more instances to the backup and have it sit waiting again. This would be massive overkill and would cost me a fortune!

Does anyone have any suggestions on how I can manage this better?

Thanks!

Answer 1

Steven - sounds like you need to spend a little time looking at your setup and also you need to consider cost vs. availability.

Azure VMs support auto-scale via the Cloud Service into which they are deployed and use the Cloud Service Autoscale capabilities to drive provisioning of new instances (which would have to be able to auto-configure themselves). A good overview can be found on the Azure documentation website.

If you find you are returning errors before you've scaled then you need to set a lower threshold for your scale trigger (lower CPU threshold for instance) or run an N+1 configuration where N is your minimum number of VMs for non-load usage scenarios. This is to reduce the TTSO of your API.

You will never hit an instantaneous scale if you don't have an already running unit available.

Finally, Traffic Manager can help spread load only where you use least-latency routing which means running different instances of your API in different Azure geographies. If that's not something you need then Traffic Manager isn't the fix.

Answer 2

Full disclosure: I am Lars Larsson, Software Architect at Elastisys AB.

What you are describing is exactly what Elastisys cloud platform can help you do: it collects monitoring data and can predictively scale up to meet demand as it arrives, not just react when your service is already suffering. The algorithms are based on solid research performed at the Distributed Systems group at Umeå University, Sweden.

However, there is no support for interfacing with Azure yet (support for AWS, OpenStack, and CityCloud is available on our GitHub page).

Please contact Elastisys if you would be willing to serve as a use case for us as we build Azure support into future versions of our software.