Deploy our own software using Puppet?

Question

(Apologies in advance for the stupidity in this question. I'm normally a programmer, not a sysadmin, but I've taken it upon myself to automate some things, and clean up some other things which are automated but not in the prettiest way. :-)

I've been looking around at various tools for automation of software deployment to a bunch of servers, like cfengine, Puppet, and Chef. So far, Puppet looks the most appealing, but I've certainly not committed to anything yet.

These tools all look like they can do a great job of keeping a bunch of servers up-to-date with prepackaged software.

What I don't get is: how does one use a tool (like Puppet) to manage deployments of our own internal software? I think I'm at a loss because I've seen a thousand tutorials showing how to keep Apache ensure => latest (which is pretty cool), but nothing that quite corresponds to my use-case today, which is something more like:

1. when a human being pushes The Button,
2. pull branch A from the version-control repository B
3. run command C to compile it
4. copy the binaries D to servers E1 through E10
5. on each server, run command F to make all changes take effect

Puppet sounds great, and I totally see the advantage of declarative, idempotent configuration over some shell scripts, but I've not seen any tutorials for "you want to update your shell scripts to Puppet (or Chef, or cfengine) so here's what you should do". Is there such a thing? Is it obvious to other people how to take the things provided in the Puppet docs and replicate the behavior I want? Am I just not getting it?

What it's sounding like to me, so far, is that the human being (#1) would manually package the software (#2 and #3) external to Puppet, manually update the Puppet config, which would trigger Puppet to update the servers ... maybe? (I'm a little confused here, as I'm sure you can tell.)

Thanks!

Answer 1

We use puppet, but we don't use it for our application deployments. As you said, you could package your software into debs or rpms, configure your private repository everywhere, and use puppet to control versions, but you're still at the mercy of waiting for the next 30 minute refresh on all your servers.

What I would do (and this is close to what we do, but we use rails so there's no compile step):

• Use puppet to configure everything on the server except the application itself. Dependencies, web servers, users, paths, etc.
• Have your automated build server (bamboo, hudson, cruise control, etc.) put the compiled artefacts in a repository manager like Nexus.
• Use capistrano to push build to your servers.

Chef might have more real-time push capabilities; I'm not very familiar with it.

Answer 2

Steps 1 through 3 are commonly automated in a build process. Normally, the output of this process will go through a test cycle. I package the output so that it can be deployed to an integration test environment. Only if the integration tests pass should steps 4 and 5 occur.

Your step 5 implies a deployment outage. For something like apache, this can be handled by shutdown and restart during log rotation. A crontab script can handle this. If you can handle rolling changes over a period of an hour or so just include the restart in the deployment step 4. Puppet or cfengine are appropriate tools for step 4. This can be triggered by updating the repository when the integration tests pass.

Answer 3

Search for puppet recipes and you will find tons of production ready scripts. Yes you would have to manually package the software.If you are maintaining your own personal repository then you could use the ensure=>latest flag. Then write a recipe to tell puppet to install the software. The recipe would need to be placed on the master server from where it would be propagated to the slaves.