Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have script scheduled with cron that transfers a file around to other machines using scp filename user@host:~ and today have noticed that sometimes the md5 of the files don't match. Upon closer inspection it seems the file is corrupt.

My question is, what tool can I use to ensure a non-corrupt transfer?

share|improve this question
add comment

2 Answers

up vote 4 down vote accepted

You can use rsync. Rsync is more reliable because:

rsync always verifies that each transferred file was correctly reconstructed on the receiving side by checking its whole-file checksum

This description is under the -c flag in man rsync. You don't have to actually use this option.

share|improve this answer
1  
I dont' think this works as desired. "-c, --checksum:This changes the way rsync checks if the files have been changed and are in need of a transfer.Without this option, rsync uses a “quick check” that (by default) checks if each file’s size and time of last modification match between the sender and receiver."This seems to indicate that the -c flag tells rsync to use a cehcksum to determine if a file has changed. It does not appear to say that the file will be checksummed after transfer to ensure non-corruption. notemagnet.blogspot.com/2009/10/… –  jdw Sep 28 '11 at 14:10
    
"Note that rsync always verifies that each transferred file was correctly reconstructed on the receiving side by checking its whole-file checksum, but that automatic after-the-transfer veri-fication has nothing to do with this option's before-the-trans-fer "Does this file need to be updated?" check." - So either way, rsync is better for transfers. –  Luke Sep 28 '11 at 14:20
    
Agreed. Rsync is the better tool for this, just the -c flag is not required for this solution. –  jdw Sep 28 '11 at 14:25
add comment

You should use a strong hashing utility that use SHA256 or SHA512 algorithms. See: MD5 collision vulnerabilities

share|improve this answer
    
Do you really consider MD5 collisions an issue when it comes to dealing non-intentional corruptions? –  andol Sep 28 '11 at 15:32
    
You shouldn't state such things so imperatively I'd say. md5 is quite ok for variety of appliance. As they say the better is enemy of just good. Tools are to be chosen adequatively to goal. –  poige Sep 28 '11 at 15:36
    
The question was how to ensure a non-corrupt transfer. Think that you are doing this for a plane computer. Would you wait for the first MD5 collision that will trigger a disaster? –  Mircea Vutcovici Sep 30 '11 at 15:00
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.