3

I have about twenty servers with different webapps. Every 4 hours runs rsnapshot task and backs up all of them to a backup server.

Accidentally today I discovered that backup failed last 4 days due to input/output failure in file system. fsck fixed the issue, however 4 backup days are lost.

Is there any way to check if backups are ok?

Right now I use munin monitoring system, if it does matter, though it check only server health (memory, cpu, hdd, etc) without any software checks. I can integrate a script that will check a FATAL ERRORs entries in rsnapshot log, however I'm not sure will it be enough?

May be there is a system for bootstrapping environment from backup to check its integrity. Unfortunately I didn't find enough information about it.

2
  • Normally, the best practice is to verify your backups after they've been written to their destination media. This is done by comparing the source with what's on the destination. I'm not quote sure how that applies in the case of rsnapshot, though.
    – ewwhite
    Aug 20, 2012 at 12:12
  • @BrentPabst fixed =)
    – rush
    Aug 20, 2012 at 12:15

1 Answer 1

4

Ensure you are also monitoring your filesystem free space, monitor system logs for critical / severe messages, SMART output for your disks, network and backup services (ssh / rsync).

Regarding verifying your backups, you may want to setup your webapps environment in parallel and recover your backup periodically. Your backups are as good as your recovery.

2
  • 1
    pretty solid advance.. you want a disaster recovery plan in place that you practice every now and then. It will not only help you get back up quicker but it will also test your backups.
    – Mike
    Aug 20, 2012 at 12:23
  • +1. I tend to think "backup procedures" as "restore procedures". Backups are worthless if they cannot be restored in a quick and reliable manner. Aug 20, 2012 at 12:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .