Generating/managing config files for hosted application

Question

Edit - I'm starting to look into Perl's Template-Toolkit. It seems to fit the bill nicely for the problem described below. Has anyone played with it? Any (detailed) advice?

I asked a question about config management, and haven't seen a reply. It's possible my question was too vague, so let's get down to brass tacks. Here's the process we follow when onboarding a new customer instance into our hosted application : how would you manage this? I'm leaning towards a Perl script to populate templates to generate shell scripts, config files, XML config files, etc. Looking briefly at CFengine and Chef, it seems like they're not going to reduce the amount of work, because I'd still have to manually specify all of the changes/edits within the tool. Doesn't seem to be much of a gain over touching the config files directly.

1. We add a stanza to the main config file for the core (3rd-party) application. This stanza has values that
   • defines the instance (customer) name
   • the TCP listener port for this instance (not one currently used)
   • the DB2 database name (serial numeric identifier, already exists, they get prestaged for us by the DBAs)
   • three sub-config files, by name - they need to be created from 3 templates and be named after the instance
2. The sub-config files define:
   • The filepath for the DB2 volumes
   • The filepath for the storage of objects
   • The filepath for just one of the DB2 volumes (yes, redundant to the first item.
3. We run some application commands, start the instance
4. We do some LDAP thingies (make an OU for the instance, etc.)
5. We add a stanza to the config file for our security listener that acts as a passthrough to LDAP
   • instance name
   • LDAP OU
   • TCP port for instance
   • DB2 database name
6. We restart the security listener (off-hours), change the main config file from item 1, stop and restart the instance. It is now authenticating via LDAP.
7. We add the stop and start commands for this instance to the HA failover scripts.
8. We import an XML config file into the instance that defines things for the actual application for the customer - user names, groups, permissions, and business rules. The XML is supplied by the implementation team.

Now, we configure the dataloading application

6. We add a stanza to the existing top-level config file that points to a new customer-level config file. The new customer-level config file includes:
   • the instance (customer) name
   • the DB2 database name
   • arbitrary number of sub-config files, by name
7. Each of the sub-config files defines:
   • filepaths to the directories for ingestion, feedback, backup, and failure
   • those filepaths have a common path to a customer-specific folder, and then one folder for each sub-config file
8. Each of those filepaths needs to be created
9. We need to add this customer instance to our monitoring scripts that confirm the proper processes are running and can be logged into. Of course, those monitoring config files include the instance name, the TCP port, the DB2 database name, etc.
10. There's also a reporting application that needs to be configured for the new instance. You get the idea.
11. There's also XML that is loaded into WAS by the middleware team. We give them the values for them to plug into the XML - they could very easily hand us the template and we could give them back completed XML.

Answer 1

I think you could use Bcfg2 very successfully for this. It is a configuration management system that is incredibly flexible and extendable. It comes with Genshi for basic templating logic, but allows arbitrary inline Python code to do more complicated stuff.

You could have a single database that contains all information about every instance, and then set up Bcfg2 to generate every bit of configuration based on that. As an added bonus, if you ever have to change something across a large number of instances (or all instances), it will be totally painless.

Answer 2

This looks very custom, so I would look into making my own tool with Fabric for the backend to run all this and Django for the web frontend.

With fabric as backend you can also use it in ssh without django.