How can one check file integrity when hundreds of files change each day? [closed]

Question

At The Tao of Backup website, the writer makes the case that one should check to see which files have changed between backups to avoid backing up corrupt files. Or, in his words, "You are then free to investigate the changes and restore any damaged files as you see fit."

I can't see how it is possible to do that when a simple search with Windows shows that >430 files have changed in just 24 hours. Though I can guess that things like %userprofile%\AppData\LocalLow\Microsoft\CryptnetUrlCache aren't important, I have a hard time being confident about that. After all, who's to say that a wiley cracker can't figure out a bug in [insert name of favorite browser here] that turns a corrupted cache file into an exploit?

Despite this problem, I can't ignore the suggestion to check file integrity because it's not hard to conclude that backing up a screwed up version of a file doesn't give you any real backup.

How can I make sure the files I'm backing up aren't in ruins without resorting to devoting the rest of my life to learning the ins and outs of every single program that ever runs on the computer and all their associated file formats?

Pre-emptive thought:

If I only check files in places like Documents, then that doesn't save me from backing up screwed up copies of say, Visual Studio. And I have no desire to reinstall all my software from DVDs/painfully slow downloads the moment the hard drive fails.

Answer 1

If you are at this level of paranoia (no backup strategy can guarantee 100% safety) you could also argue that the corrupting program/cracker is also giving back files to you that are not corrupt, while you are checking them. It corrupts only the backups.

The only option I can think of then, is to archive some backups and manually pick out the one that is not corrupted (yet).

I think the best strategy would be to avoid that corruption problem: Namely avoid intrusions and viruses and use a filesystem that can detect bit rot like ZFS or Btrfs.

Answer 2

Bit rot is completely different kind of corruption than unauthorized change; bit rot is trivially detectable, so I leave it out for now.

If you mean this advice I don't consider it particularly realistic. If a virus or malicious user is changing your files, "intelligently" and slowly, the same files that you also change from time to time, no automated solution can help you. No ECC, no MD5 sum, no structural validation, no backup (!) guarantees that you would keep all your data exactly "true". The only sure remedy is to periodically compare all your stored data with the data that you subjectively trust - such as your brain, brains of other people, physical measurements, paper document, etc.

Neat huh? In other words, if you are writing a book (that is by editing a file), the only way to be completely sure it hasn't been maliciously corrupted at any occasion is to verify it manually with your own memory.

This is why this risk is usually ignored.

Only with static files this problem is easily solved with a decent backup. Just restore the Visual Studio (static) binaries from the backup that you've created just after installation/upgrade.