Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a couple of servers (HP ProLiant DL380, DL360 and DL120), which are all set up with hot spares, as well as a couple of regular workstations with some external disks.

I want to create preventive maintenance routines for these systems, and was wondering which maintenance would be advisable.

  1. For the servers with the hot spares, is it sufficient to replace the hard drive upon failure?

  2. For regular workstations and external disks, is there any check I can do to check the status of the hard drives, or possibly some interval at which I want to replace the hard drives anyway? Or is it simply recovery from backup upon failure that is done?

share|improve this question
    
I have also read that the number of bad sectors on the hard drive increases over time. Is there any threshold when it would be a good idea to get a new hard drive? (I assume it is easy to count the number of such bad sectors) –  tor Aug 16 '12 at 12:06

2 Answers 2

up vote 1 down vote accepted

I managed thousands of HP Prolient servers with various RAID setups in a large datacenter (64,000 sqft). There are usually 2 situations in which i'd replace drives, upon failure or upon predictive failure. However, the latter IMHO is much like "at some point in the future this drive is going to fail." I've heard it has something to do with comm or monitoring issues with HP.

I would suggest you always replace a disk if it has failed even if the server has a hot spare. I hate using this phrase but think murphy's law. Yet a few things come to mind.

  1. Level of support with HP. Do you have a contract for support with them?
  2. How critical is the box?
  3. Field tech accessibility. Can a CE get there quickly?
  4. Part availability from HP.
  5. A rather big issue is department budget. Can you afford to replace them?

Again, these are just a few things that come to mind.

As for preventative measures, a good clean environment with operating temps and humidity within ASHRAE (a quick blog post here) or manufacture specs is about the best you can do. Also, good monitoring goes a long way to catching these issues as they happen to minimize downtime. You could take a look at Hp Insight. Personally, i stay away from the hp agents and strictly monitor servers from their integrated lights out (iLo) interface with help from HP SIM and an in-house ticketing system.

As for workstations, the above can still apply. You should never be in a situation where you have to recover from backup. And simply replacing disks to replace them is no way to run a server/workstation. Then again, i don't manage workstations so some of this may be overkill.

share|improve this answer

For the servers with the hot spares, is it sufficient to replace the hard drive upon failure?

So these are probably the same model of disk, perhaps even from the same manufacturing batch. How confident are you that each one will fail at different times? OK, a standby disk should have a very different workload which will help to stagger it's failures relative to the disks holding real data - which obviously must be in a RAID array - but does your RAID array support failure of more than 1 disk?

i.e. have you got monitoring and alerting in place to detect a failed disk even when the filesystem is still avalable?

You can proactively monitor disk health by trapping SMART alerts - but you doidn't say what operating system is running on these. Running periodic surface scans is a good idea too (frequency depends on how criticial the data is and how much impact there would be from a data loss - but not more than once a month). It should be posible to swap out the disk from the RAID set then re-integrate it with no downtime. Indeed if you do it right then no performance impact.

share|improve this answer
    
They are mostly Windows 2003 R2 servers. I am neither confident that they will fail at different times, or that our surveillance of the state here is sufficient. The computers support a fairly large RAID array, but I currently only use two disks. –  tor Aug 16 '12 at 11:44

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.