Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

So I was in the shower this morning thinking about stuff, and I had an idea which may be terrible or may be brilliant I'm not sure.

On a server, install git

cd /
git init
git add .
git commit -a -m "Yes, this is server"

Then get /.git/ to point to a network drive (SAN, NFS, Samba whatever) or different disk. Use a cron job every hour/day etc. to update the changes. The .git directory would contain a versioned copy of all the server files (excluding the useless/complicated ones like /proc, /dev etc.)

For a non-important development server where I don't want the hassle/cost of setting it up on a proper backup system, and where backups would only be for convenience (I.E. we don't need to backup this server but it would save some time if things went wrong), could this be a valid backup solution or will it just fall over in a big pile of poop?

share|improve this question
2  
doesn't sparkleshare using similar idea?? –  B14D3 Dec 15 '11 at 14:17
    
@B14D3 I think sparkleshare is more of a sort of dropbox type thingy, but I'll look into it –  sam Dec 15 '11 at 14:20
1  
you're right, but it using git to make some sort of buckup thing (copying to several pc's and controling versions of files);) –  B14D3 Dec 15 '11 at 14:28
    
The big problem with this is that there is no central control - you need to have direct (ssh) access to the machine to preform any form of maintenance or backup validation. I always find installing an app on the boxes to be backed up then administering them from a central location is a much bigger win. –  hafichuk Dec 15 '11 at 16:03
    
@hafichuk With tools like Puppet/Chef it's not such a big issue, but I see your point. –  sam Dec 16 '11 at 10:45
add comment

11 Answers

You're not a silly person. Using git as a backup mechanism can be attractive, and despite what other folks have said, git works just fine with binary files. Read this page from the Git Book for more information on this topic. Basically, since git is not using a delta storage mechanism, it doesn't really care what your files look like (but the utility of git diff is pretty low for binary files with a stock configuration).

The biggest issue with using git for backup is that it does not preserve most filesystem metadata. Specifically, git does not record:

  • file groups
  • file owners
  • file permissions (other than "is this executable")
  • extended attributes

You can solve this by writing tools to record this information explicitly into your repository, but it can be tricky to get this right.

A Google search for git backup metadata yields a number of results that appear to be worth reading (including some tools that already attempt to compensate for the issues I've raised here).

etckeeper was developed for backuping up /etc and solves many of these problems.

share|improve this answer
6  
+1 for mentioning ACLs/permissions –  Larry Silverman Dec 20 '11 at 17:59
5  
Git also doesn't store empty directories. –  Flimm Nov 22 '12 at 11:05
    
and it also sucks for tracking file moving / renaming, through history. –  Cawas May 10 '13 at 23:19
add comment

It can be a valid backup solution, etckeeper is based on this idea. But keep an eye on the .git directory persmissions otherwise pushing /etc/shadow can be readble in the .git directory.

share|improve this answer
add comment

I've not used it, but you might look at bup which is a backup tool based on git.

share|improve this answer
    
Never seen bup before, looks interesting –  sam Dec 15 '11 at 13:59
    
I've started using bup recently, just a few days before my hard drive crashed ;) Restore went fine, so recommended! –  André Paramés Dec 16 '11 at 14:30
add comment

Whilst technically you could do this I would put two caveats against it:

1, You are using a source version control system for binary data. You are therefore using it for something that it was not designed for.

2, I worry about your development process if you don't have a process (documentation or automated) for building a new machine. What if you got hit buy a bus, who would know what to do and what was important?

Disaster recovery is important, however its better to automate (script) the setup of a new development box than just backup everything. Sure use git for your script/documentation but not for every file on a computer.

share|improve this answer
4  
Development boxes all come from KickStart files, and actually the average box lasts for about 2 or 3 months before it's re-built. But people change configs and do things, we re-build the boxes and people say "hey, I know I didn't put it in source control but I had some shit on that box" and I laugh at them for being stupid. All around, good times. Binary data would be a bitch, it's something I totally overlooked while in the shower. –  sam Dec 15 '11 at 13:56
    
I applaud your attitude to those that fail to follow basic principals. Personally I have a similar situation to you, however I have a git repository which links in all the config files that might be important rather than a catch all. Plus a txt doc with setup steps. –  Phil Hannent Dec 15 '11 at 14:01
1  
I think git works pretty well for binary files, vide Google Android's bulk part of the repo are git repositories of prebuilt executables. –  user377178 Dec 22 '12 at 22:11
add comment

Well it's not a bad idea, but I think there is 2 red flags to be raised:

  • If the harddisk fail, you'll lose everything if you're not pushing your commit to another server/drive. ( Event if you've a plan for it, I prefer to mention. )

... but still, it can be a good backup for corruptions-related things. Or like you said, if the .git/ folder is somewhere else.

  • This backup will always increase in size. There's no pruning or rotation or anything by default.

... So you may need to tell your cronjob to add tags, and then make sure commit that are not tagged will be cleaned up.

share|improve this answer
    
We would probably mount the .git directory on a remote server, although the clasic rm -Rf / would cause us some issues. Our current backup system keeps stuff for 2 years or 50 versions (whichever comes last) so our backup is constantly increasing anyway. But I like the idea of adding tags, we could have "daily", "weekly" etc. tags –  sam Dec 15 '11 at 13:58
    
+1 for ever growing space requirements –  hafichuk Dec 15 '11 at 16:00
    
@sam git is ever growing. You canot prune the history older than N years. I suppose your current system does. –  rds Dec 16 '11 at 10:44
1  
Regarding increase in size, please do 'git gc' regularly or before you push to another (central) server. Without this the git repo may grow (much) larger than it should. I once had a 346 MB git repo that can shrink down to 16 MB. –  Hendy Irawan Feb 13 '12 at 14:27
add comment

I haven't tried it with a full system but I'm using it for my MySQL backups (with the --skip-extended-insert option) and it has really worked well for me.

You're going to run into problem with binary data files (their entire contents could and will change) and you might have problems with the .git folder getting really large. I would recommend setting up a .gitignore file and only backing up text files that you really know you need.

share|improve this answer
    
I'm using it for MySQL backups too, with --extended-insert=false. Be sure to "git gc" regularly or right after commit. –  Hendy Irawan Feb 13 '12 at 14:32
add comment

I found this to be a good methodology for my dev boxes. It changes them from being something that needs to be backed up to only a deployment endpoint.

All the configuration and package installation manifests are stored in Puppet, allowing for easy redeployment and configuration updates. The Puppet directory is backed up with git. Kickstart is used to do the initial deploy.

I also keep a custom YUM repository for whatever packages are being developed at the time. This has the added benefit that whatever packages we are working with aren't just left as unattended binaries on the local system - if that happens and the files get nuked oh well. Someone didn't follow proper procedure.

share|improve this answer
add comment

I had the same idea to backup with git, basically because it allows versioned backups. Then I saw rdiff-backup, which provides that functionality (and much more). It has a really nice user interface (look at the CLI options). I'm quite happy with that. The --remove-older-than 2W is pretty cool. It allows you to just delete versions older than 2 weeks. rdiff-backup stores only diffs of files.

share|improve this answer
add comment

You might want to check out bup on github which was designed to serve the purpose of using git for backup.

share|improve this answer
add comment

My personal opinion is that this is basically all backwards. You're pushing the files into a backup solution, rather than pulling them out.

Much better would be to centralise the configuration of the server in the first place, and then pull it down, using something like puppet.

That said, it may work, i just dont think it'd be that good.

Try looking into backuppc - its pretty easy to set up and is frankly brilliant.

share|improve this answer
add comment

I am extremely new to git, but aren't branches local by default, and must be pushed explicitly to remote repositories? This was an unpleasant and unexpected surprise. After all, don't I want all of my local repo to be 'backed up' to the server? Reading the git book:

Your local branches aren’t automatically synchronized to the remotes you write to — you have to explicitly push the branches you want to share. That way, you can use private branches for work you don’t want to share, and push up only the topic branches you want to collaborate on.

To me this meant that those local branches, like other non-git files on my local machine, are at risk of being lost unless backed up regularly by some non-git means. I do this anyway, but it broke my assumptions about git 'backing up everything' in my repo. I'd love clarification on this!

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.