Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a 4 disk 5TB raid5 setup where a disk is showing signs of going down the drain. It is reporting media errors and from dmesg I can see that several read errors are corrected. smartctl does report "notifications" but no panic so far. Since new disks are rather expensive at the moment I am starting to pondering exactly how robust the linux md layer is.

I would appreciate if someone could shed some light on how md actually deals with disk errors. For example how does md deal with write and read errors - what does it (really) take for disk to be rejected from an array. I also read that recently md got support for mapping out bad blocks. Does this mean that the read errors I've had would have been mapped out if I where running kernel >3.1 or would md still try to "work on them" to make them usable.

share|improve this question
add comment

3 Answers 3

up vote 1 down vote accepted

If you care for your data, change the disk now. Every subsystem does everything it can to prevent data loss, but it can only do so much. Even it can map around bad blocks: When there is one, there will be others

You got massive warnings about imminent problems: Ignoring them is simply irresponsible for your clients data.

Edit, since this is too long for a comment:

I have no idea what your question really is? Do you want to know if the warnings are serious? Yes, they are. Do you need to something about it? Yes, and now.

When some subsystem reports reads error, they are there and they are important. Even if the system is still capable of correcting them, there is a good chance that this isn't the case anymore two minutes from now and the disk will be marked as failed and taken offline.

Linux md raid is at least as trustworthy as any hardware RAID implementation, but I would consider a RAID5 a dangerous choice in the best of situations. The problem with it is that you will be unprotected during a restore, which is a very taxing situation for a RAID5. I consider it more trustworthy than some hardware RAID systems, as you will get advanced warnings when a disk is about to fail, while a hardware RAID might inform you only after the fact. There are some drawbacks to a software RAID, mostly related to performance issues, but reliability is not one of them.

Anyway, regardless of this, the professional approach to a situations like yours is to replace the disk immediately, and that would me my reaction for my private disk array as well (I have a spare lying around for this kind of situations).

share|improve this answer
    
I am aware that the best thing is to change the disk. Thank you for the advice, but this is not answering the question and for some reason you are assuming that the data in question is not my own and that is simply not true. Thanks for your concern anyway. –  Waxhead Nov 26 '11 at 22:28
    
I implicitly answered the question: Trust the warnings and replace the disk. If this is your own data, you are more likely than not no pro sysadmin, and therefore the question is off topic here, as per the faq. –  SvW Nov 26 '11 at 22:32
1  
Respectfully: I do not agree with you. I am asking how md deals with write and read errors and you have not answered that. Also as far as I can understand I have the right to ask questions here since I am responsible for a Debian server at work thus making me a sysadmin to my best knowledge. I do not run a raid on that machine and the raid in question is my own. However the responses I get here helps me decide if a software raid is sufficient for the computer at work. I hope this justify me asking questions here. –  Waxhead Nov 26 '11 at 22:40
    
@Waxhead: See my edit. –  SvW Nov 26 '11 at 23:12
2  
@Waxhead: The Linux block I/O system is layered, and if a read or write error reach the level of the md system, it will consider the disk dead because the specialized sub systems handling the actual I/O operations already gave up all hope and reported the error to the higher levels. There is simply no point for the md system to try anything beyond that. As a matter of fact, even the disk itself (which really nows best) tried everything to recover before even reporting any problems to the I/O drivers. –  SvW Nov 26 '11 at 23:52
show 1 more comment

This is from a about 6 years old experience with one of our Dell-servers: back then we did not use hardware raid, since there was no possibility to remotely detect disk failures.

So we used software raid 1 (md). A little time later the raid degraded. Looking at /var/log/messages I saw that there were a list of IO errors related to a certain partition.

I re-added the partition to the raid and a short time later it got thrown out again.

I sent Dell-support that /var/log/messages output and got a new disk at once. This was on an unsupported Linux-flavour and only with the most basic support level.

We had a couple of other machines with set up and never ran into these problems again (i.e. the disks never failed). To me this is a proof that you can rely on md.

I think this still holds true for 3.1 kernel (I don`t have any of these yet).

share|improve this answer
add comment

As far as i know mdadm is a robust software RAID implementation that with HD in good status rarely generates errors of any type, i have several server with mdadm configured and i never got read errors in the log files.

Check the status of your array with :

mdadm --detail /dev/mdX

However i strongly suggest you to replace the drive responsible of errors ...

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.