I'm interested in using Rackspace Cloud Servers API to build myself a dynamically scalable hosting service. The Cloud Servers API allows me to add and remove server instances programmatically.
System Requirements
There would be a single shared server in most cases, but it might also be a dedicated server and upon bandwidth exceeding a certain target, a new server would be provisioned, with the web directories replicated or restored from somewhere (could use Cloud Files, or svn, git, etc). Could use puppet or some custom script for configuration files.
Ideally, each domain or group of domains would have bandwidth quotas that once exceeded would trigger the deployment of a new server with only the files for that domain for a period of 1 hour. If bandwidth is still exceeded after that target, the server would remain in effect another hour, and so on until it was no longer necessary at which point it would be removed from service.
If anyone has already deployed such a system or can provide some assistance doing so, it would be very appreciated!
Key questions:
1. How can I determine the bandwidth usage across multiple instances?
2. How can I avoid having DNS caching issues when a given server is only up for an hour (can I use a proxy server here)?
3. What is the best way to ensure fast deployment of files to each newly deployed server?
4. Using a dynamic cluster like this, how can I support FTP ( I prefer not to use FTP, but some projects might require it).