I'm running a high traffic service on AWS/Scalr.
The application side has nginx as load balancer with app server(s) downstream. It is managed by scalr including dns updates.
When the traffic gets high a 2nd app server (apache/php) starts and handles the traffic nicely. However after a while both servers load gets to 0 and the 2nd server shuts down. At this point the 1st server gets high load and causes a new 2nd server to start.
This way I get servers starting and stopping every half hour.
What are the general guidelines to do such scaling properly and how can I find what is the bottleneck that causes the 1st server to choke although it can handle the amount of traffic normally.
The scaling params are: scale up if 5 min load avaerage goes above 25. scale down if it goes below 2. Normally the servers have less than 1 load average.
