Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm setting up a NAS box based around freenas and ZFS.

I've read lots of posts (like this one) about "deep recovery" and green drives when using RAID-5.

Does ZFS (vs Raid-5) mean that this problem goes away, or should I still be looking at Red or Black drives to put into my NAS?

share|improve this question
How important is your data? – Michael Hampton Feb 13 '13 at 19:21
@phatmanace, this is a great example of why you don't accept the first answer that gets posted if it's not something that you go out and verify yourself. It's usually a good idea to wait a day or two to let the community vote on things before you accept an incorrect answer :) – MDMarra Feb 13 '13 at 19:44
it was the second answer chonologically. Actually I found what he had to say pretty useful, especially the links. I accepted it because it, and the links in it answered my question. – phatmanace Feb 13 '13 at 19:53
I suppose even a wrong answer is an answer... – Michael Hampton Feb 13 '13 at 21:49
potentially - yes - if it leads you to more information that helps answer the question from your perspective. – phatmanace Feb 13 '13 at 22:30

WDC Green drives have the "deep recovery" problem. You'll need Red or RE drives to avoid it.

I have a ZFS RAIDZ of Green drives at home. They've lasted almost 3 years of Power On Hours without a single error. This may be just lucky, but errors don't generally happen all that often. So you have to ask, is the cost difference worth it. Take the value of up-time multiply by the likelihood of failure, now you have the amount of additional money you should spend on tech that would mitigate that potential failure mode. In most business situations, the answer is going to be a clear yes, as the cost difference is fairly small.

share|improve this answer

I think the real issue is will ZFS have a problem with TLER? I can't speak from personal experience as I have yet to encounter the issue. The FreeNAS forums has some good discussion on this matter. I think this post is most appropriate in responding to your question. The best paragraph that answers your question:

So does TLER matter for ZFS? This answer to TLER really is based on the hardware and less so the software to the best of my knowledge. As long as your hard drives aren't being dropped out of the array and as long as the hard drive will not try to read the sector in perpetuity then you should be okay. As long as you are accepting the fact that when a bad sector is found and you're watching a movie that movie may freeze for 30 seconds or so all will be fine.

I found another thread on the FreeBSD ZFS forums that also discusses TLER's role. This post looks to answer your question as well.

With ZFS, you want to let the drive do it's thing instead of returning early because of TLER. That feature is only useful for RAID controllers that decide to panic and do strange things (like dropping a drive prematurely) if a drive takes its time doing recovery. When a drive is connected to a 'dumb' HBA, as recommended with ZFS, then the FreeBSD kernel is in control of these timeouts and ZFS will just wait for the drive to recover from any media failure without really caring how long this will take. I believe the maximum drive timeout is then dictated by the FreeBSD disk controller driver together with the default timeout of the drive itself.

So yes, I'd consider TLER irrelevant when not using a special RAID controller.

So after reading these posts, my personal opinion is that hardware controllers do play a factor (albeit a minor one). If you're doing JBOD and let ZFS get direct access to the disks, I believe TLER can be dealt with. What I'm unsure of is how well ZFS can respond depending on the RAID configuration. I'm curious to know if RAID-Z2 will be more difficult to deal with TLER, then say RAID-0 or 1, or 10 because of the additional time for parity calculation.

Edit: (forgot to answer question)

Does ZFS (vs Raid-5) mean that this problem goes away, or should I still be looking at Red or Black drives to put into my NAS?

I think ZFS can deal with TLER, but ZFS may end up dropping the disk from the array to let it finish repairing. I'd also think that firmware versions matter for each disk vendor but that's another question in and of itself. My opinion: if you're using ZFS with a supported HBA or RAID controller in JBOD/passthrough, I think you'll be fine choosing either WD Red or Black. I'm just installing a pair of WD Reds in my ZFS array so if anything happens, I'll update this post but I think the key idea to take away is you should be okay as long as ZFS has direct access to the disks.

share|improve this answer
Great answer - thanks. Followup question I guess is if ZFS does drop the drive out of the array, and sometime later the drives firmware has repaired/corrected the error - can you manually add the disk back into the pool without rebuilding it? – phatmanace Feb 13 '13 at 17:35
I'd say yes (to rebuilding) if you use RAIDZ/Z2/Z3 as parity is calculated. I'd bet doing a mirror (RAID1) or 1+0, there would be little to no rebuilding as there's no parity calculation involved, but if a rebuild needed to happen, I would not be shocked to see the time to rebuild dramatically reduced for a mirror in comparison to a RAIDZ configuration. – osij2is Feb 13 '13 at 18:50
-1 Sorry. The problem with those forums is that way too many people, who don't know what they're talking about, weight in with opinions. Drives dropping out of arrays is annoying, but certainly the array continues to function. When a drive sits and does "deep recovery" for 30 second, the array stops working for those 30 second, including ZFS. Worse, the OS may try the same sector multiple times. It's usually not just one sector that goes bad either, it's quite a few. Every time one of these sectors is read the array stops for 30 seconds - bad news in a production business environment. – Chris S Feb 13 '13 at 19:06
I totally agree with your comment on production business environment, however, the OPs question was asking if the TLER issue goes away in ZFS RAIDZ. I personally haven't experienced the issue and since no one has definitively stated "yes" or "no", we're left with whatever info (preferably reputable) we can find. I posted from relatively reputable boards (FreeNAS, FreeBSD) so I tend to trust their information a little more than others especially regarding the subject matter. Is the information I posted inaccurate or misleading? Please feel free to correct my answer. – osij2is Feb 13 '13 at 19:19
The parts that you filled yourself are mostly just begging more questions. Then you "answer" with quotes from people who are mostly wrong. TLER doesn't matter to ZFS so much as to the user who will be waiting for 30 second on every HD error without it. While ZFS is happy to passthough the delay to the user, it's likely preferable to the user that major delays do drop a disk (and thus alleviate the delay). TLER is certainly not useful only to RAID controllers. Also, the Black drives don't have TLER; the Red and RE do. – Chris S Feb 13 '13 at 19:31

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.