Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have quite a few FTP folders, and I add a few each month and prefer to leave some sort of method of verifying their integrity, for example the files MD5SUMS, SHA256SUMS, ... which I could create using a script. Take for example:

find ./ -type f -exec md5sum $1 {} \;

This works fine, but when I run it each time for each shaxxx sum afterwards, it creates a sum of the MD5SUMs file which is really not wanted.

Is there a simpler way, or script, or common way of hashing all the files in to their sums file without causing problems like that? I could really use a better option.

share|improve this question
up vote 2 down vote accepted

Are you saying the problem is you are re-running the md5sum on the generated file? You could just skip those files. And of course, use gnu parallel to speed things up:

find . -type f -a \! -name MD5SUMS | parallel -j+0 "md5sum {} >>MD5SUMS"

I feel however from your description that I'm missing something though.

EDIT: corrected redirection and added xargs info:

Note you don't have to use parallel, xargs works fine too (I just think it's fun to try parallel). Here's the equivalent xargs invocation:

find . -type f -a \! -name MD5SUMS -print0 | xargs -0 sum >> MD5SUMS
share|improve this answer
Edit: I think you are right, although I do seem to not have parallel on my Ubuntu 10.10 server. – Kennie R. Feb 14 '11 at 6:03
Bah, installed parallel but it seems to only sum one file and list the others nonsummed, strange. – Kennie R. Feb 14 '11 at 6:16
Sorry I typoed the command, the > overwrites the file every time. Needs to be >>. – Phil Hollenback Feb 14 '11 at 6:53
Ah, excellent - thank you. The xargs one barfs at filenames with spaces in them, that is unfortunate, but the parallel one works fine. Hey , Learned a neat tool, all the better! – Kennie R. Feb 14 '11 at 7:11
I fixed the xargs invocation to work with files with spaces in them by using -print0/-0. – Phil Hollenback Feb 14 '11 at 7:24

I've had a need for verifying integrity of backups/mirrors which contain a large number of files and ended up writing a command-line program called MassHash. It's written in Python. A GTK+ Launcher is also available. You may want to check it out...

share|improve this answer
Good idea but in it's current form, I tried to set it up but got "OSError: [Errno 2] No such file or directory", despite being able to get the GUI up fine. Btw better installation instructions could be: wget; python (done) – Luke Stanley Oct 23 '12 at 5:38

Try md5deep

sudo apt-get install md5deep
md5deep -rel "test_directory" > results_file.md5

"This is the command I will run against the directory to check for any changes."

md5deep -X list.txt -r Pictures/

From :

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.