9

I am looking for a solution for backing up about 30 of my servers that are a mix of centos, windows 2003 and 2008 machines.

I have a san in another data center with a 1 Gig link between the sites that im going to be backing up onto so this is purely a disk based backup solution.

What I am ideally looking for is some open source software possible with a nice gui that will allow me to have a backup cycle with full backups on a monthly basis that go back 6 months and weekly backups that are saved for one month. I'm hoping for de duplication as well so if a file is saved on the first backup the software creates a symlink to it rather then saving the data again in all the proceeding backups. It must also be smart enough to know when a file is deleted that the file is moved to the latest backup it exists in and all symlinks to it are updated.

I was looking at bacula but im unsure if it can do the deduplication or something similar. If it can will someone be kind enough to point me at a decent guide as to how to do it.

If it does not can someone recommend another solution open source or not?

1
  • I think bacula has deduplication, but it might only be available with the enterprise version of it. It was causing us some issues so our team decided to turn it off to fulfill a security requirement. Dec 3, 2015 at 22:41

6 Answers 6

9

Something simple like rsnapshot may do what you want. It doesn't sound like you want what's traditionally referred to as deduplication. The synthetic full backup functionality using hard-links that rsnapshot provides sounds like exactly what you're asking for.

(On the traditional deduplication front you may want to look at the Opendedup project as a starting point. It's the most complete open source data deduplication project that I've seen. It uses a custom filesystem (SDFS) as the basis of its functionality. I tend to think that this isn't want you're looking for, but in the interests of completeness I mention it here.)

1
  • Rsnapshot does seem like the sort of software im after. Is there any decent gui tools where i can setup scheduled snaphots from a centralised location? I'm thinking of having monthly backups that i retain for 1 year and weeklies that i retain for a month.
    – user71173
    Feb 19, 2011 at 15:44
3

Bacula writing to a ZFS filesystem that has de-duplication turned on, then write the snapshot from that host to a remote host with ZFS and apply that transaction log to that filesystem.

4
  • 2
    sounds massively over complicated. From what i'm reading about backup the most important thing is K.I.S.S
    – user71173
    Feb 19, 2011 at 15:50
  • @user71137 -- the question was how to do opensource backups with deduplication, not how to draw a line with a ruler. Besides, anyone who tells you backups are simple is someone who has never had to do restores.
    – chris
    Feb 22, 2011 at 3:09
  • 1
    this is exactly why you should try to keep things simple, because restores never are.
    – Chris S
    Jun 10, 2011 at 2:43
  • ZFS is surprisingly simple to use, but you will need to understand the underlying operating system (FreeBSD, OpenIndiana/Illumos, etc.). There are appliances and distributions which can help (Such as ixsystems.com/freenas). Backula is relatively simple when compared with enterprise backup software like NetBackup. Jun 17, 2011 at 21:01
2

I have used backuppc with good success on a range of windows linux and mac machines. http://backuppc.sourceforge.net/ http://www.zmanda.com/backuppc.html (info on De-Duplication)

1

Here few links for de-duplication of top of my head:

  • OpenDedup - de-duplicating file system that works on Linux and Windows.

  • s3fs - de-duplicated file system backed by Amazon S3 service.

1

I ended up going with synbak http://www.initzero.it/products/opensource/synbak/ as it does the hardlinked copies using rsync and can pull them from remote servers unlike rsnapshot.

I'm also looking into using something like rwsync or deltacopy to setup rsyncd on windows machines so I can use synbak for windows machines as well.

1

You now have Obnam which does that also, and much more !

1

Not the answer you're looking for? Browse other questions tagged .