85
votes

In an environment with multiple system administrators, I see a few advantages to adding the server config files into a revision control system. Most notable are the ability to track changes, who made them, and of course being able to roll back to known working configs.

I'm mainly interested in Unix/Linux solutions, but would be curious to Windows implementations as well.

1

12 Answers 12

52
votes

I have tested this at home (~ 3 hosts) for some time now, trying different scms (RCS, Subversion, git). The setup that works perfectly for me right now is git with the setgitperms hook.

Things you need to consider:

Handling of file permissions and ownership

  • RCS: does this natively
  • Subversion: last I tried, you needed a wrapper around svn to do this
  • git: the setgitperms hook handles this transparently (needs a fairly recent version of git with support for post-checkout hooks, though)

Also, if you don't want to all of your /etc under version control, but only the files that you actually modified (like me), you'll need an scm that supports this kind of use.

  • RCS: works only on single files anyway.
  • Subversion: I found this to be tricky.
  • git: no probem, put "*" in the top-level .gitignore file and add only those files you want using git add --force

Finally, there are some problematic directories under /etc where packages can drop config snippets that are then read by some program or daemon (/etc/cron.d, /etc/modprobe.d, etc.). Some of these programs are smart enough to ignore RCS files (e.g. cron), some are not (e.g. modprobe). Same thing with .svn directories. Again a big plus for git (only creates one top-level .git directory).

4
28
votes

I've done it informally with git, but there's also the etckeeper project which is a more completist and detailed implementation.

1
  • 4
    etckeeper is really good - it handles restoring permissions (not supported by git, hg, etc) and supports your backend of choice (including git, hg, bazaar, etc). Also has integration into APT so that every time you do an apt-get operation, the /etc repository is committed, and does overnight commits. I've used this for a while and overall it's much better than using a vanilla VCS, if only for the permissions feature.
    – RichVel
    Apr 13, 2012 at 9:19
23
votes

Another option is to use an automated server configuration tool like Puppet or Cfengine to script your server configurations in a declarative language.

It's extra work on the front-end, but using a utility like Puppet allows you to automatically rebuild and configure a server with very little human intervention.

1
  • 5
    Yes, but you should also revision-control your Puppet/CFengine configs. I'm also a fan of revision-controlling the output so that you can answer the question "what was the config on date x?" as well as "what should the config have been according to puppet?", and correlate inputs with outputs for troubleshooting the config management system. May 7, 2009 at 0:34
10
votes

I have been experimenting with etckeeper which seems to work pretty well. I doesn't require a centralized server, which may be important in some situations. You can use several different DVCS backends, so you can choose the one you are most familiar with. It seems to work very well for me, but I haven't tried getting the other techs where I work to start using it yet.

6
votes

I have been looking into Chef lately. Not only does it keep templatable (.erb) configs in version control, but allows you to perform actions (like restarting a service after you uploaded the configs to the node). Chef helps with package management so you can verify dependencies with any node you interface with (i.e. has to have sudo package installed). Chef seems to be easily extensible in Ruby, so if you have any custom processes you can just script it out within the framework provided.

But still have not tried it and you do have to install Ruby on the client and server with the appropriate gems (this really isn't that hard). Overall looks really easy to manage many servers at once.

1
  • We use Chef heavily (60+ servers) quite sucessfully. All recipes and config files are checked into Subversion. May 31, 2011 at 20:21
3
votes

I am in the process of implementing Puppet across our infrastructure, and it is very conducive to keeping its data in version control.

I prefer Mercurial since it's just a collection of files with some metadata stored in hidden directories (easy to manage, easy to understand, easy to use).

My Puppet files are at /usr/local/etc/puppet/ (FreeBSD 7.1). All it took to add Mercurial to it:

> cd /usr/local/etc/puppet
> hg init

All changes are committed with a simple "hg commit." If a change hoses something, I can roll back every single server to a given version of the file (say, sudoers) with a single command.

Great intro to Mercurial

3
votes

I have been using Subversion on the servers I manage. Works fine. I have also set up a Trac instance, so we have a timeline view, ticketing system, browsing, etc.

Using symlinks, cron and subversion I have also setup automated configuration distribution based on the subversion repository, where every Linux server updates a repository using svn update with scripts (e.g. firewall scripts).

2
votes

Here's a real life use case: Used Subversion to manage configuration files on 4 different servers. I'd recommend using version control for configuration files for the same reason you would use them with code - it's a backup and an undo button all in one. If I were managing a much larger amount of servers and they were much closer in terms on configuration I'd be using something like Puppet as detailed in berberich's answer.

The idea is that you can have one repository that you can checkout specific folders on the servers (eg. /var/named/) so I both have a history and a backup of configuration files (the backup is a bonus if you make the mistake of using a GUI configuration application that wipes your hand edited additions cough Server Admin in Mac OS X Server cough). It's then easy to test it on a test server and subsequently update the production server with files that work without manually copying files.

1
vote

I've created a project a few years ago to do exactly this: Savon

It uses subversion to store files, and has some additional features, like tracking ownership, permissions, and SELinux context. It also allows you to logically split your file system changes in layers, so you can for example track changes that should go to all your web servers separately.

0
votes

Subversion is very easy to setup and use and there are a lot of resources:

Basic How-To

SVN Book

Document Management Overview

1
  • I agree, if you are using windows checkout tortise svn, its very simple to use.
    – Element
    May 6, 2009 at 18:09
0
votes

Most of our changes are managed with our Help Desk system, even for routine maintenance type stuff. We have been slowly moving our documentation into a wiki for our own use, and what we publish to end users. Posting the configuration changes and the discussion behind it, is nice to have open on our intranet.

0
votes

For many years I used rcs for files I started modifying, but a couple of years ago I started putting the whole /etc under git control. It requires some work to check in files in granular bulks (some times I resort to a huge "various updates" checkin), and I have written some scripts to help with this, but etckeeper mentioned seems very interesting, I will try out immediately.

Not the answer you're looking for? Browse other questions tagged .