0

I have quite a few FTP folders, and I add a few each month and prefer to leave some sort of method of verifying their integrity, for example the files MD5SUMS, SHA256SUMS, ... which I could create using a script. Take for example:

find ./ -type f -exec md5sum $1 {} \;

This works fine, but when I run it each time for each shaxxx sum afterwards, it creates a sum of the MD5SUMs file which is really not wanted.

Is there a simpler way, or script, or common way of hashing all the files in to their sums file without causing problems like that? I could really use a better option.

Improve this question

3 Answers 3

Reset to default
2

Are you saying the problem is you are re-running the md5sum on the generated file? You could just skip those files. And of course, use gnu parallel to speed things up:

find . -type f -a \! -name MD5SUMS | parallel -j+0 "md5sum {} >>MD5SUMS"

I feel however from your description that I'm missing something though.

EDIT: corrected redirection and added xargs info:

Note you don't have to use parallel, xargs works fine too (I just think it's fun to try parallel). Here's the equivalent xargs invocation:

find . -type f -a \! -name MD5SUMS -print0 | xargs -0 sum >> MD5SUMS
Improve this answer
6
  • Edit: I think you are right, although I do seem to not have parallel on my Ubuntu 10.10 server.
    – Kennie R.
    Feb 14, 2011 at 6:03
  • Bah, installed parallel but it seems to only sum one file and list the others nonsummed, strange.
    – Kennie R.
    Feb 14, 2011 at 6:16
  • Sorry I typoed the command, the > overwrites the file every time. Needs to be >>.
    – Phil Hollenback
    Feb 14, 2011 at 6:53
  • Ah, excellent - thank you. The xargs one barfs at filenames with spaces in them, that is unfortunate, but the parallel one works fine. Hey , Learned a neat tool, all the better!
    – Kennie R.
    Feb 14, 2011 at 7:11
  • I fixed the xargs invocation to work with files with spaces in them by using -print0/-0.
    – Phil Hollenback
    Feb 14, 2011 at 7:24
0

I've had a need for verifying integrity of backups/mirrors which contain a large number of files and ended up writing a command-line program called MassHash. It's written in Python. A GTK+ Launcher is also available. You may want to check it out...

http://code.google.com/p/masshash/

Improve this answer
1
  • Good idea but in it's current form, I tried to set it up but got "OSError: [Errno 2] No such file or directory", despite being able to get the GUI up fine. Btw better installation instructions could be: wget example.com/example.py; python example.py (done)
    – Luke Stanley
    Oct 23, 2012 at 5:38
0

Try md5deep

sudo apt-get install md5deep
md5deep -rel "test_directory" > results_file.md5

"This is the command I will run against the directory to check for any changes."

md5deep -X list.txt -r Pictures/

From http://linhost.info/2010/05/compare-hashes-with-md5deep-part-2/ :

Improve this answer
1
  • Unable to install on amazon linux. sudo yum install md5deep gives this error - No package md5deep available. Error: Nothing to do
    – Sandeepan Nath
    Oct 21, 2016 at 9:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .