10

Mean time between failures may be difficult to interpret, but there is a wealth of statistical method that you can use if you have some hard data.

Trouble is, nobody reports their MTBF numbers anymore. (Other than hard drive manufacturers, anyway.)

Where do you go to find MTBF data for components and servers?

1
  • I'm curious to know how you're using MTBF data.
    – dr.pooter
    Jun 15, 2009 at 5:39

6 Answers 6

2

Why MTBF doesn't matter

The mean time between failure number isn't as important as the uncorrectable error rate. MTBF deals with the complete failure of the part, read the drive. However that number is meaningless when a single bit in error will cause a RAID 5 panic and bring the hot spare into play.

While the MTBF for professional and consumer level drives has increased by an order of magnitude in recent years, the uncorrectable error rate has stayed relatively constant. This rate is estimated at 10^14 bits, so one bit per 12 terabytes read, for consumer SATA drives, source.

Why you should loose sleep over your RAID 5 array

So, that is only 6 passes of a brand spanking new 2Tb drive. How long does it take to read 12Tb of data? A lot less time than the MTBF for that drive.

http://storagemojo.com/2008/02/18/latent-sector-errors-in-disk-drives/

What is more concerning is the chance of a double read failure on a RAID 5 array consisting of drives that large. With a 7 1Tb drive RAID 5 array, the probably of a second read failures while doing a RAID rebuild, is 50%.

http://blogs.zdnet.com/storage/?p=162

4
  • You could always use RAID6 perhaps?
    – Chopper3
    Jun 15, 2009 at 9:29
  • 3
    Great answer, but only covers hard drives Jun 15, 2009 at 9:30
  • @Chopper3, yes RAID6 does improve the situation, but once you've dedicated two disks to parity, and a third to hot spare, then on a 7 drive array, you're getting pretty close to the same space as a RAID10 array. Jun 15, 2009 at 9:33
  • I'm looking for data for more than just hard drives. Entire servers do still fail from time to time, so it's worth measuring how often.
    – user9495
    Jun 17, 2009 at 10:13
2

It is a shame that people think that the MTBF Figures don't apply to complex systems. The real problem (afaik) is that the manufacturers don't have the MTBF figures for their hardware modules. These are figures that should by all rights be available. Dell saying "Dell no longer lists specific MTBF's for their servers." is actually atrocious! They may as well say "Well our stuff is really not reliable enough to be used where an MTBF figure is required".

The reliability engineer (or guy wearing the RE's hat) is supposed to limit the scope of the availability study. This is often limited to the hardware modules.

As for the classification of what constitutes a failure... Well that's why we perform an FMECA analysis.

Sure systems are complex, and failure modes include software failures, but that's often not the scope of the study. We want MTBF Figures for hardware. Ask your salesman to provide this. It is their technical responsibility to provide it to you... If they refuse or side step, go somewhere that has telecom grade servers with mandated availability figures for hardware.

1
  • The problem when a vendor has to publish the MTBF is that they have to publish it sooner than they can collect real data. Hence they need to produce the MTBF through some sort of extrapolation. Sometimes that can be way off. The worst case I have seen was off by more than three orders of magnitude.
    – kasperd
    Apr 21, 2016 at 6:39
0

I have seen MTBF reported on company support sites. Talk with your sales person or SE to obtain the information.

0

In my view, MTBF numbers have become a sales tool. Modern hardware has reached a state where MTBF numbers are essentially useless. Even the lowest of the low-ball vendors is producing hardware that outlasts any sensible upgrade cycle. As you note, nobody's reporting MTBF numbers. I believe this is the reason.

2
  • And yet, some servers are still more reliable than others. We do need to answer questions like "is a second power supply worth it?" For that we need data. Ideally, that would be real failure stats reported across a population of like devices. We use MTBF as a weak proxy for that actual distribution.
    – user9495
    Jun 17, 2009 at 10:15
  • Fair enough. In my little world, the idea of redundancy is an expected part of the process. For another example, look at most large-scale hosting providers, or google. I still suggest that given the commodity status of wintel servers, this is a waning issue. If you're talking about z-series or similar, the equations and expectations are much different.
    – dr.pooter
    Jun 17, 2009 at 13:21
0

Unfortunately, MTBF is not a practical or reliable measurement in modern servers. The all concept of MTBF is that if a specific model / configuration is being used by many over long time, we can likely know its reliability.

Today, most of us happily trade potential extra reliability for proved extra performance and power efficiency. For example, would you build your new servers on 18-24 months old hardware just because it proved its reliablility? or just go with the last generation of CPUs with more cores, horsepower and power efficiency?

Also, unlike old-school telephony systems, systems are quite customized, and of course, heavily reliant on software. How reliable is BIOS version x.xx or driver version y.yyy? Is the latest OS/DB/app server patches increases stability or does it have stability regressions? How many servers in the world actually use the same exact mixture of hardware/ stack version as you?

If you need high availability, you will anyway need to add redundancy to your system (dual-everything, clustering, hot spares, DRP, what have you). So, the relative reliability of each hardware component is typically not a significant factor, as you build you infrastructure to survive single components failures. Just live with the uncertainty (reliability is retroactive) and plan accordingly.

2
  • The problem of constantly-changing configurations is a real one. That makes it difficult to build up a body of experience with a single configuration point. Nevertheless, if you are planning for HA, even with a redundant configuration, you must have some notion of the individual devices' reliability.
    – user9495
    Jun 17, 2009 at 10:16
  • It seems there is no hope for IT to ever become a science. We keep working on assumptions, no hard data, and waste of resources. More like black magic than anything these days. Engineering seems a distant goal.
    – gtirloni
    Jun 11, 2013 at 6:09
0

I agree with most of the other answers: MTBF numbers aren't useful to me and I never check them.

The one exception is hard drives, but even there, I only look at MTBF in a very rough way, being sure to buy the more reliable "server-class" drives if there's a choice.

You must log in to answer this question.