We manage our more-or-less standard set of systems to do the usual hosting business:
- authoritative dns servers
- Mail eXchangers (with the first stage of spam filtering)
- mail scanning server (amavisd doing content scan)
- mailbox server (where the actual mailboxes reside, running IMAP/POP/webmail)
- mysql server
- web server
we already are slowly implementing managed configuration of the systems via puppet, but the next step will be to automate the service lifecycle. Currently we have an in-house developed solution that basically consists of cronjobs on the various servers that read data from MySQL and deploy a new domain:
- creates zone in DNS and put records in it pointing to our servers
- enables relay to that domain in our MXes
- add optional custom config for domain to amavisd
- creates mail domain on mailbox server
- creates empty db in MySQL with corresponding privileges
- creates directory structure, ftp credentials, apache vhost on the web server
99.9% of the times we add domains, 0.1% of the times we remove them or shuffle them around in some way - the latter is currently done by hand via ssh. Some systems are already spread over multiple servers (e.g. domains spread over more than one mailbox server) and some will be in the future (eg. MySQL).
We're gonna develop a simple web interface for our customers to manage their domains and get new ones and we obviously would like to eliminate all human work wherever possible. The web interface (programmers, think view) will not contain any business logic and will talk (probably via RESTful-like HTTP) to another application that'll do just that (programmers, think controller).
Would you interface the business logic app to puppet and use that to manage the services? Or let's talk directly with the systems? (eg, mysql config is done via SQL, proftpd accounts + mail relay data + antispam settings are stored in the db, apache vhosts could be handled by a simple agent that could do create/config/purge and similar stuff)?
How do the big hosting shops manage this kind of stuff? Also, any other advice would be welcome.