up vote 3 down vote favorite
share [g+] share [fb]

I have a very small and quite old hard drive disk, about 32G.
On to this disk I have copied a largish tar file, about 5G.

When I run md5sum to generate a checksum on this file I repeatedly get different results (on the same machine and the same file). This obviously should not happen.

If I repeat the experiment with a much smaller file, as expected the checksum is the same each time. I can only assume that because the large file is spanning most of the disk, and it is an old drive, I am experiencing a lot of read errors on the hard drive - and it needs replacing? Could there be any other good reason for this? Something I can do to fix the problem other than buying a new disk?

Update: sha1sum also produces inconsistent results.

link|improve this question
82% accept rate
4  
This file isn't /dev/random, right? :D – Kevin M Jul 7 '09 at 12:26
No! BTW - this is on a linux (debian) machine. – Joel Jul 7 '09 at 12:28
3  
DO NOT HESITATE. Replace this drive immediately! – kmarsh Jul 7 '09 at 12:31
6  
Browsing thru the questions this morning and coming upon this gave me a chuckle. The title reads something like "Smoke is pouring out of the windows of my house. Do you think I should do something?" I hate to make light of someone else's misfortune, but it just struck me as funny. I'm sick, I guess. – Evan Anderson Jul 7 '09 at 13:13
1  
Yes, it's a crappy machine. I think it was stored outside for a while too. I can see rust on it. Infact - I'm always a little scared to touch it incase I get an electric shock. – Joel Jul 7 '09 at 13:23
show 1 more comment
feedback

8 Answers

up vote 5 down vote accepted

It may be worth running fsck (or the equivalent for whatever filesystem you're using) to fix disk errors. But, it could be the last thing this drive does if it's on its last leg.

link|improve this answer
going for it - running fsck.... – Joel Jul 7 '09 at 12:35
actually seems to have fixed it! – Joel Jul 7 '09 at 16:38
1  
You should still replace the drive ASAP. – SilentW Jul 7 '09 at 18:29
1  
He did say it was an small, old drive. But, if it were worth keeping you can check out Spinrite to do an analysis and repair. It can tell you how damaged the drive is. – spoulson Jul 7 '09 at 18:49
feedback
up vote 6 down vote

The only likely reasons are bad disk or bad RAM, and as the disk is old and you are not experiencing other issues it is less likely to be the RAM.

It could be a dodgey connector/controller: I once had an IDE->USB adaptor go bad resulting in corrupted transfers even thought the drive was perfectly fine.

link|improve this answer
I, too, have seen the both the bad IDE connector/cable and the dodgy IDE to USB adaptor. Both situations nearly gave me heart attacks, but both were completely benign. – Evan Anderson Jul 7 '09 at 13:14
feedback
up vote 4 down vote

To be on the safe side I suggest you run memtest86+ over night to make sure that it finds no memory errors.

link|improve this answer
feedback
up vote 1 down vote

Does sha1sum have the same behavior?

I would expect disk read errors before receiving phantom data that alters the hashing process. What platform are you running on?

link|improve this answer
Running on Linux - Debian. – Joel Jul 7 '09 at 12:28
feedback
up vote 0 down vote

Since data is different each time it is possible that md5sum is able to read less and less after each run. But failed reads should cause md5sum to quit and report input/output error.

It is very important to backup whatever data you have on this drive and shift to new drive. This drive can fail any time. Do not use this drive for any important work.

link|improve this answer
feedback
up vote 0 down vote

Is there any process or cron job that is set to automatically add or remove files from this tar file?

I'd also run a disk check. Make sure, as you said, that there aren't any problems with the disk itself.

link|improve this answer
feedback
up vote 0 down vote

I once had this same problem as well. It turned out to be the hard drive controller on the motherboard.

However, in this case it definitely sounds like a dud hard drive. Hard drives tend to hang onto life by marking sectors bad, transparently, in the background. When sectors finally start to go bad at an OS level the drive ends up expiring very quickly.

link|improve this answer
feedback
up vote 0 down vote

Same happening here. doing "md5sum", "sha1sum" and "openssl rmd160" on the same file (which is not opened by any process) and it gives me different results ALMOST every time:

livecd ~ # while [[ 1 ]]; do md5sum stage3-i686-20100608.tar.bz2; done
26f6ca4969f33fe08dc51a15a739268f  stage3-i686-20100608.tar.bz2
30f494ed6a0310eb1b916497e90b5c03  stage3-i686-20100608.tar.bz2
e70dfe75622fc30829332e4161517d42  stage3-i686-20100608.tar.bz2
f6e7189e98fa4580da52e6b11e244c56  stage3-i686-20100608.tar.bz2
26f6ca4969f33fe08dc51a15a739268f  stage3-i686-20100608.tar.bz2
f612b0880cdd4ca787019cd1a2d9b791  stage3-i686-20100608.tar.bz2
df5b37cea0e11e07f29267db9d9f942b  stage3-i686-20100608.tar.bz2
26f6ca4969f33fe08dc51a15a739268f  stage3-i686-20100608.tar.bz2
ac1327b0a448a038a72e99e2556daead  stage3-i686-20100608.tar.bz2
26f6ca4969f33fe08dc51a15a739268f  stage3-i686-20100608.tar.bz2
26f6ca4969f33fe08dc51a15a739268f  stage3-i686-20100608.tar.bz2

The real md5 sum of the file is 26f6ca4969f33fe08dc51a15a739268f, which you can see that it appears a couple of times.

I'm working on a Gentoo Live CD without any filesystem mounted (other than the CD image), and the file is stored in a ramdisk, so no disk is being used in my case :-(.

Also, I let memtest86+ do 9 complete passes on the machine and havent found any errors.

If somebody can give me a hint on what else to try, it will be REALLY appreciated... I'm lost here.

link|improve this answer
1  
This should be in it's own question, there an "Ask Question" button in the top right. Please do not hijack old questions just because they are similar. Welcome to ServerFault! You're likely to get good answers to your specific questions if you use the "Ask Question" button in the top right. – Chris S♦ Jun 9 '10 at 17:41
Tell me how the problem I'm having is different from the problem of the original poster, quote: "md5sum repeatedly gives different checksum for same file on same machine". If you want me to open a new thread, I'll be happy to do so.. but don't try to convince me that my problem is unrelated. Best regards, Germán. – German Jun 9 '10 at 17:57
@German - As it is now you are not lightly to get any answers to your question. Not because there is something wrong with your question but simply because no one is looking for questions posted as answers to another question. And especially when the question is almost a year old. My suggestion is that you copy the entirety of your post into a new question as Chris suggested. – Nifle Jun 9 '10 at 18:33
@German Unlike forums, this site has a 'Question/Answer' layout. So your response is seen as an Answer to @Joel's question when in reality you're looking for an answer to your own question. If you want an answer to the question you've posed, you need to click on the Ask a Question button and ask it there. As others have pointed out, no one is going to look at this because it's not a question. – George Stocker Jun 9 '10 at 19:45
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.