I'm developing a web application (django/gunicorn/nginx) that needs to be scaled out according to load. The app will be hosted on Linodes so I intend to use StackScripts (and maybe Puppet) to start up new instances of the web server, then stick them behind a NodeBalancer.
It looks as if Nagios and Munin will allow me to monitor load, get alerts when the server is under pressure, and view some pretty graphs. But will those applications also allow me to automate the deployment of a new Linode? It looks as if I should be able to write a Nagios event handler that launches the StackScript. But I'm not sure it's possible to create a check that can determine when to start new instance.
- Is it possible to set up an alert that takes past measurements into account? My criteria will be based on how long a machine is under load, rather than an instantaneous reading. I'm not worried if the web server is close to max usage for one check but I may be if it stays that way for two or more.
- Am I missing a piece? I'm thinking this is possible in a plugin that uses data already available to Nagios/Munin. But maybe I need to write a separate app or script that stores previous check values and does the comparisons.
- Can anyone point me to an example of using Nagios to scale out an app? I've seen a few slideshows where people discuss scaling this way (usually on EC2) but no concrete examples.