Currently I back up my personal files using a simple shell script that creates a tarball. It is being run by cron hourly.

I'm planning to switch to incremental backups (I will use duplicity), because this will minimize the data transfer over network.

This leads me to some general questions:

  • when making incremental backups, what strategy to choose? As I understand, such a strategy implies creating a full backup, then some incremental backups. At some point, a full backup is made again and it starts a new set. My problem is I don't know how to decide how often I should make this full backup again. If I'm doing backups hourly, would it make sense to start the day with a full backup and do incremental ones later, or is there a better strategy?
  • how many full sets of backups should I keep before starting to rotate them?
  • What are your retention and restoration goals? Your backup strategy really isn't something you get off the shelf like potato chips, you should think through how much data you could stomach losing, how long it should take to restore, how much knowledge is required to restore (you may not be the one doing the restore...). Personally; I take full weekly and monthly backups. The monthly are archived on tape in my safe; then twice daily a differential is taken since the weekly. I keep 6 months worth on disk; years on tape so far (DDS3 tapes are cheap). – Chris S Sep 13 '10 at 3:04
  • It's my personal files (about 100 MB, ~30MB compressed) so nobody else will be dealing with them. It's a one man show. For now I'm keeping 10 full copies. I don't need 6 months worth, because most of these files is source code in version control, and it contains all the history. – halp Sep 13 '10 at 3:15
  • If you get hit by a bus, you wont be the one doing restores... just a though. If 10 fulls has been working for you, I'd stick with that. I don't see much reason to do a full more often than weekly unless you have some off-site requirement; but I'd ditch incremental backups and do differential. The restore complexity would be significantly less, especially if you're taking them every hour. And the additional size is negligible in your case. – Chris S Sep 13 '10 at 3:22

The right answer depends on questions like:

  1. How much space is available on your backup device
  2. How long do you want to keep the backups around?
  3. How often does your data change?
  4. What is the "Rate of change" (How much data will change between full backups?) I've read a 5% rate of change is typical for many businesses. Your personal files may be more or less frequent then that.

Unless you have special needs, try to keep your schedule simple. Complex backup schedules can cause confusion during the restore, and you don't want more confusion during the restore.

Many backup products offer a simple, default backup schedule similar to the following. This can be easily adopted to your future needs.

  • Monthly fulls, keep for 1 year. Done on a Sunday.
  • Weekly fulls, keep for 1 month. Done on a Sunday.
  • Daily incrementals, keep for 2 weeks.

  • Hourly incremental sounds too frequent for most people, as the files don't change that frequently.

| improve this answer | |
  • @halp: if you want to research this type of backup further, lookup "grandfather, father, son" backups via google – Sirex Sep 13 '10 at 9:58

Incremental backups are a complete PITA and should be avoided if at all possible.

If network bandwidth really is an issue, then I'd recommend maintaining a mirror elsewhere and replicating from the source using rsync/unison then creating a coherent tarball of the refreshed image on the destination.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.