Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I just rebooted my monitoring server for the first time in a while, and the following starting filling the screen:

Jul 11 23:52:30 monit kernel: [   25.255908] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
Jul 11 23:52:30 monit kernel: [   25.256170] ata1.00: BMDMA stat 0x24
Jul 11 23:52:30 monit kernel: [   25.256278] ata1.00: failed command: READ DMA
Jul 11 23:52:30 monit kernel: [   25.256410] ata1.00: cmd c8/00:c0:20:68:35/00:00:00:00:00/e0 tag 0 dma 98304 in
Jul 11 23:52:30 monit kernel: [   25.256416]          res 51/40:9f:41:68:35/00:00:00:00:00/e0 Emask 0x9 (media error)
Jul 11 23:52:30 monit kernel: [   25.256809] ata1.00: status: { DRDY ERR }
Jul 11 23:52:30 monit kernel: [   25.256933] ata1.00: error: { UNC }
Jul 11 23:52:30 monit kernel: [   25.304388] ata1.00: configured for UDMA/66
Jul 11 23:52:30 monit kernel: [   25.304430] ata1: EH complete

. . . 

Jul 11 23:52:30 monit kernel: [   25.552451] sd 0:0:0:0: [sda] Unhandled sense code
Jul 11 23:52:30 monit kernel: [   25.552462] sd 0:0:0:0: [sda]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Jul 11 23:52:30 monit kernel: [   25.552475] sd 0:0:0:0: [sda]  Sense Key : Medium Error [current] [descriptor]
Jul 11 23:52:30 monit kernel: [   25.552490] Descriptor sense data with sense descriptors (in hex):
Jul 11 23:52:30 monit kernel: [   25.552498]         72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00 
Jul 11 23:52:30 monit kernel: [   25.552529]         00 35 68 41 
Jul 11 23:52:30 monit kernel: [   25.552543] sd 0:0:0:0: [sda]  Add. Sense: Unrecovered read error - auto reallocate failed
Jul 11 23:52:30 monit kernel: [   25.552559] sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 35 68 20 00 00 c0 00
Jul 11 23:52:30 monit kernel: [   25.552587] end_request: I/O error, dev sda, sector 3500097
Jul 11 23:52:30 monit kernel: [   25.556607] ata1: EH complete

I already know I need to replace the HDD (Cost of Data > Cost of HDD), but I want to know for my own knowledge what's actually wrong with it.

Yes, our monitoring server has no RAID, just one HDD... Don't look at me...

share|improve this question
1  
"Yes, our monitoring server has no RAID, just one HDD... Don't look at me..." -- not looking too much, if you have good backups. –  Janne Pikkarainen Jul 12 '12 at 6:39
2  
I hope you have good backups. You're probably going to need them. If the monitoring server is that important, the cost of replacing the hard drive with at least a RAID 1 should be easily justifiable. –  Michael Hampton Jul 12 '12 at 6:39
1  
Even if you have no RAID, you should be monitoring SMART data for all your disks. –  James Youngman Jul 12 '12 at 7:05

4 Answers 4

up vote 11 down vote accepted
sd 0:0:0:0: [sda]  Add. Sense: Unrecovered read error - auto reallocate failed

Looks like the drive has bad sectors and is unable to reallocate these (possibly because it's run out of spare sectors). The output of smartctl -a /dev/sda would give you more information on the state of the drive.

share|improve this answer

Lassie's saying "arf! arf arf! arf!". Which is dumb, because this has nothing to do with Timmy or wells. This is why you don't take sysadmin advice from dogs.

The drive is giving you an "Unrecovered read error - auto reallocate failed", which basically means "I tried to read, I failed, I tried to recover (read the sector a few more times, apply some ECC, and move the data to a sector that isn't broken), and it didn't work". This probably means (as mgorven says) that the disk is chock full of reallocated sectors already, because the disk's been dying for a while, but I also think it can mean that it wasn't able to recover the sector at all (repeated reads + ECC failed to get a good-looking data block).

Either way, yeah, the drive's very, very cactus. Your data isn't looking real healthy, either.

share|improve this answer
1  
Right - the SMART data may also tell you how many sectors were reallocated already. –  James Youngman Jul 12 '12 at 7:05

Make a dd image or rsync copy of that disk now++, unless you have a full backup allowing a convenient restore of that box. And start looking for a compatible and working replacement disk.

BTW, UDMA/66, is that a ten year old PATA disk?

share|improve this answer
2  
DD will also try to read the broken sector(s). –  Hennes Jul 12 '12 at 8:18
1  
Usually not to the best effect ... there is an advanced version called ddrescue with better control over error behaviour. –  rackandboneman Jul 12 '12 at 9:40
    
There's always conv=noerror, but I'm not sure how well that works on what will then become a broken file system. Probably not worse than reading the data file by file would, but certainly not any better... –  Michael Kjörling Jul 12 '12 at 13:41

I know this is old, but just in case someone is still reading this post: "DD will also try to read the broken sector(s)" - gddrescue is useful here. It doesn't (okay, it does, but only once).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.