Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I found from this posting on Stackoverflow, if you just wanted to get a list of corresponding files whose content differs, then you could execute:

diff  --brief --recursive dir1/ dir2/

...and trying this myself, this gives a nice little list of files whose content is different.

But if I have made a backup of an original filesystem (assume BIG filesystems, 2TB for example), and I wanted to check if the backup is the same as the original (eg. I want to check if the backup is successful), what should I run, if I am only interested in which files are NOT available on either side? I want something that checks directory trees FAST (not checking contents).

Although comparing the filesize does not give an absolute indication on whether the backed-up file is different, comparing that and other non-filecontent metadata during the comparison command would help.

(My apologies for reposting a very similar question, but I noticed a user arguing that the question was perhaps off-topic, so would this question definitely be on stackoverflow.)

Any experience about this?

share|improve this question
4  
take a look at rsync with dryrun and verbose options? –  lVlint67 Sep 13 '13 at 17:10

3 Answers 3

up vote 3 down vote accepted

A non-rsync solution that checks modification time and file size:

diff <(find dir1 -mindepth 1 -printf '%P %t %s\n' | sort) <(find dir2 -mindepth 1 -printf '%P %t %s\n' | sort) | grep -E '^[<>]'

Example output:

> bar Sat Sep 14 11:52:33.0544384394 2013 91
< foo Fri Sep 13 17:58:21.0213671425 2013 445
> foo Sun Sep  8 23:01:02.0355432619 2013 347

Shows that foo has been modified in dir1 and bar only exists in dir2.

share|improve this answer
    
This is great, but I'm only interested in seeing the files that are different? What would the diff flags for that be? I had a look at man diff, and I wanted something --brief, but it only gave me a message saying that the dirs were different –  Florian Mertens Sep 14 '13 at 15:18
    
Answer edited: I added output cleanup to the command, and an output example. –  Changaco Sep 14 '13 at 21:33

You don't tell us what backup solution you use, but almost all real backup programs have a "verify" mode. (A "real" backup tool in this context being something like Bacula, software expressly designed for backup and restore).
All other things being equal I would suggest using real backup software, and its built-in verification tools.


All other things NOT being equal, rsync -nav --delete <backup> <original> would be my suggestion.
This does check contents, and will tell you about extra files, different files, and missing files. It's not as in-depth (slow) as a diff, and the -n means it won't actually touch the filesystem.

share|improve this answer
    
I don't think it does check contents, rsync's default mode checks modification time & size. –  Changaco Sep 13 '13 at 18:16
    
@Changaco that is correct rsync only looks at timestamps and sizes. The question specifically mentions that checking the contents is not required. –  Zoredache Sep 13 '13 at 18:36
    
This gives an enormous output! Also: Assume no backup solution was used, just plain cp dir1 dir2. –  Florian Mertens Sep 14 '13 at 15:20
    
Also, i know this is a dry-run and your suggested command does not delete anything, but it DOES make me nervous... it's a bit like dry running rm -rf / –  Florian Mertens Sep 14 '13 at 15:22
    
@FlorianMertens If you don't trust the tool, don't use it. (If you can't trust your tools in system administration you have larger problems. I routinely run that exact command to sanity-check syncs before performing them, and I wouldn't use rsync at all if there was a chance it wouldn't understand & respect -n) –  voretaq7 Sep 16 '13 at 15:30

Resulting from the answers above, I came up with the following little script, that I wanted to share:

# vi /bin/checkbackup

#!/bin/bash
echo "1/5: Checking source files..."
find "$1" -type f -printf "%P %s\n" | sort > ~/.checkbackup_files.tmp
echo "2/5: Comparing destination files..."
find "$2" -type f -printf "%P %s\n" | sort | diff - ~/.checkbackup_files.tmp
echo "3/5: Checking source directories..."
find "$1" -type d -printf "%P\n" | sort > ~/.checkbackup_dirs.tmp
echo "4/5: Comparing destination directories..."
find "$2" -type d -printf "%P\n" | sort | diff - ~/.checkbackup_dirs.tmp
echo "5/5: Cleaning up"
rm -f ~/.checkbackup_files.tmp
rm -f ~/.checkbackup_dirs.tmp

# chmod 755 /bin/checkbackup

Usage:

$ checkbackup /path/to/dir1 /path/to/dir2

No reports means the folder structure and names are the same, and that the files have same sizes and same names, without any missing.

Tested on 1.5TB directory trees on SATA2 5400rpm discs, it finishes at approximately 5mins.

This script is by no way finished, there are undoubtedly other, better, faster, and more in depth with more checks tools out there. But I thought I would share it anyway.

share|improve this answer
    
Tip: If the report is enormous, run: $ checkbackup /path/to/dir1 /path/to/dir2 > ~/Results.txt Then: gedit ~/Results.txt to view it in a nicer editor (you can also use other command based tools, this is just how I could do it)... hope this helps anyone –  Florian Mertens Sep 15 '13 at 19:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.