Access controll strategy for production deployment

Question

We have a development model where every developer is responsible for deploying the program he/she is responsible for.

What we want to achieve is the following:

1. for all developers to be able to deploy their applications to our production environment.
2. have a good level of security
3. have accountability on who did what.
4. keep track of which application use what resources.

Currently we have written all our deployment scripts in fabric.

We have setup one account for each application. SSH-keys for that account are handed out to the developers responsible for deploying that application. This account is also used for running the application.

This achieves point 1, 4 and somewhat 2, but not 3. It also has the disadvantage that the server environment must be prepared since we don't want to provide root access to the application accounts (dependencies be installed by someone who is root, directories created).

How should we set up our deployment strategy in order to solve the above points, and have we missed some important goal for the strategy?

Answer 1

We have a similar strategy right now, and we are moving away from it, exactly because of your point 3) above.

Situation where a programmer is also responsible for releases looks really nice and simple, convenient and fast, but if you care about the stability of the system and the accountability, it quickly turns into a nightmare as it encourages quick hacks ("noone has to know, here, we fixed it quickly, like it never happened") and discourages proper documentation ("but everyone knows that"). Peer review? Fuhghehtaboutit ;)

All you will get with such a system is a perpetual fight between sysadmins and programmers, trust me, I was there not so long ago.

The only way to a stable and well documented system is that all the steps required for a release are properly documented, scripted and tested as often as possible, ideally the test systems should be automatically rebuilt every day. Once you are at that stage, anyone can run the scripts and do the release, and it should not be the one who wrote them, as he has no interest in reporting his own omissions.