3

I had a similar question a few months ago Good backup/archiving software?

Right now i have been using Norton Ghost but i notice if i move large files (100mb+) it will make another copy of it. It eats up gbs like crazy, i need something else. Do you guys have any suggestions?

7 Answers 7

3

Backuppc might be your answer. -- http://backuppc.sourceforge.net/

From the doc:

Identical Files

BackupPC pools identical files using hardlinks. By ``identical files'' we mean files with identical contents, not necessary the same permissions, ownership or modification time. Two files might have different permissions, ownership, or modification time but will still be pooled whenever the contents are identical. This is possible since BackupPC stores the file meta-data (permissions, ownership, and modification time) separately from the file contents.

0

If you use backup software that uses Single-instance storage such as Windows Home Server or Danz Retrocspect (now EMC Retrospect) it shouldn't matter f you move your files around or even duplicates them, since the backup software will identify identical files and only store one copy of them.

0

Was just researching the same solution, I ran across Acronis Backup which appears to have plugin to do this but haven't yet had a chance to test this out.

0

Obviously this may not suit you as it involves certain assumptions and compromises, but one possible way of doing this is:

  • store your files in an image file, mounted as a loopback device (I use qcow2)
  • snapshot the image file (eg with VSS, LVM etc) and use rsync or equivalent to transfer

I do this to keep bandwidth low for off-site backups - I then use rsnapshot on the contents of the image file at the remote site as I am not concerned about bandwidth there.

0

For small text files you can use git. Don't know how it works with large files (100mb+).

1
  • It works quite nicely with large binary files too, provided enough bandwidth, but it is likely to have the same problem because it also handles file moves as though they were adds and deletes, while storing history. Sep 17, 2013 at 5:54
0

I know this is old, but extremely relevant as duplicating 100's of GB's to Tb's of uneeded files often is problematic for anyone budget conscious.

I have been using Areca Backup. and it seems to keep track of duplicate files well. I recently moved 300GB one day and it only backup up 8GB actual data meaning it did not recopy the files and just referenced them

My manual file mirror, versions, and deletions had bloated 350GB (300 noted above) in 3 months where ArecaBackup only bloated 20GB over the same time frame.

I appreciate it's ability to backup the files in such a way that you can access the actual file in a directory tree on the backend; that is files can be stored in a directory tree in their original format instead of a proprietary format to fight possibility of corruption of a backup. Although typically you would browse through the GUI.

  • It is free and open source GPL2.
  • It works.
  • It can output a script so you can easily run a cron job around it.
  • It supports full, differntial, and incremental backups
  • It supports compression
  • It has a Delta transfer option.
  • It has a GUI which is very nice for recovery
  • It has a plethora of options and customization you can do as well.
  • It is java based so it is a bit platform agnostic

I have not found a way to delete a file from the archive, so if there is a large file I want gone I delete it in the directory structure and if someone tried to recover it for some reason I'm sure it would throw an error when recovering it. Although I only do this for known large temporary or duplicate files so this has not been an issue; namely from users using backed-up locations as a scratch space and the scratch work ends up getting backed up.

The biggest danger is if your configuration files get corrupted or go missing you cannot rebuild them; but you still would have your files.

Overall if you do not like vendor lock in or proprietary file formats, it is a great solution.

I have no affiliation with them, they just provided something that solves a problem for me!

-1

How about using rsync for this (or a Windows equivalent)? If you use the --delete option, it will automagically remove files that are missing due to renames/deletes.

1
  • 3
    ...and if you'll occasionally delete some important file it will be deleted on backup as well.
    – rvs
    Mar 27, 2012 at 10:28