I have about twenty servers with different webapps. Every 4 hours runs rsnapshot task and backs up all of them to a backup server.

Accidentally today I discovered that backup failed last 4 days due to input/output failure in file system. fsck fixed the issue, however 4 backup days are lost.

Is there any way to check if backups are ok?

Right now I use munin monitoring system, if it does matter, though it check only server health (memory, cpu, hdd, etc) without any software checks. I can integrate a script that will check a FATAL ERRORs entries in rsnapshot log, however I'm not sure will it be enough?

May be there is a system for bootstrapping environment from backup to check its integrity. Unfortunately I didn't find enough information about it.

  • Normally, the best practice is to verify your backups after they've been written to their destination media. This is done by comparing the source with what's on the destination. I'm not quote sure how that applies in the case of rsnapshot, though. – ewwhite Aug 20 '12 at 12:12
  • @BrentPabst fixed =) – rush Aug 20 '12 at 12:15

Ensure you are also monitoring your filesystem free space, monitor system logs for critical / severe messages, SMART output for your disks, network and backup services (ssh / rsync).

Regarding verifying your backups, you may want to setup your webapps environment in parallel and recover your backup periodically. Your backups are as good as your recovery.

  • 1
    pretty solid advance.. you want a disaster recovery plan in place that you practice every now and then. It will not only help you get back up quicker but it will also test your backups. – Mike Aug 20 '12 at 12:23
  • +1. I tend to think "backup procedures" as "restore procedures". Backups are worthless if they cannot be restored in a quick and reliable manner. – Janne Pikkarainen Aug 20 '12 at 12:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.