2

I have a Dell T7500 with a PERC H710P connected to 4 3T drives in a RAID5 array. Also connected to the controller are 2 256G SSD drives, not configured in an array. A Linux server is installed on one of the SSD drives, and the RAID5 is where all my user data are stored.

The other day upon boot, the RAID BIOS reported errors

Drives 01 and 03 missing
Foreign config available

I loaded the foreign config, and the drives reappeared. On the next boot, I got

Drive 01 offline

Thinking the drive was bad, I replaced it with a new drive and rebuilt drive 01. When I next booted, the system came up OK, but a few reboots later I got

Drive 00 offline
Foreign config available

So I read in the Foreign config and forced 00 online.

After several reboots I then got

Drive 03 offline
Foreign config available

Read in foreign config. Force drive 03 online.

Now the system comes up OK. I have rebooted it many times.

Should I assume that my controller is bad?

Or said another way, is there any possibility that this kind of behavior can be caused by something other than the controller? For example, can the kernel driver muck up the driver configuration somehow?

Improve this question
0
5

Yes, I believe either your controller or the raid backplane is bad. But I think the controller is the culprit. Can you look up the firmware version of the RAID controller (not to be confused with the system BIOS, which you should also check) and compare to what is available on Dell's site? You may find the version is quite old and that critical issues have been resolved in newer versions. Alternatively you could try calling Dell support - which you should certainly do if support is available! You can easily check what service contract is in force by looking up the Service Tag at support.dell.com.

Two notes of caution. You are in dangerous territory. Upgrading the RAID controller firmware can sometimes result in data loss - make sure the new version has been out for awhile, and read the release notes carefully. 2) RAID 5 doesn't give you a lot of wiggle room. Either way prepare to back up your critical data before you let time pass on this issue or take any substantial corrective actions!

Improve this answer
3
  • 1
    RAID controller FW (and driver for compatibility) update would be a good idea before proclaiming the hardware as being bad. And I 2nd the suggestion of backing up data - RAID5 is risky business, and shouldn't really be used for critical data on drives of this size. – JimNim Nov 14 '13 at 15:23
  • I did talk to Dell. Curiously, they didn't recommend updating my bios, which is out of date. I will indeed update my bios and give that a shot. – jsp Nov 14 '13 at 15:46
  • I got a new RAID controller, and everything seems to run OK now. – jsp Nov 19 '13 at 15:59
2

In addition to the great suggestions by whitepaws (get a backup, update FW, etc) you may consider contacting Dell support if you're still under warranty. There's a chance that the controller log would have some more information about what's going on, and could clue you in on a potential cause of your issues.

In a more direct answer to your question... yes, it is feasible that something other than the controller itself could cause the issues (namely a bad driver, if yours happens to be old enough). It's not very likely though - issues with the controller itself, or with some of the other hardware are more likely. Sometimes 1 bad disk can cause issues with all of the others too - this is something we'd check for in the controller log.

Improve this answer
4
  • I talked to Dell, but they didn't mention a controller log or even suggest updating my firmware. Where do I find the log? I don't see anything in the configuration software. – jsp Nov 14 '13 at 15:52
  • 1
    The controller stores its log internally, and must be exported with a utility. Typically this is done from OMSA (which you would need to install). Do you have your case number handy? (Might want to direct-message it to me, not sure if this is an appropriate place to post it) – JimNim Nov 15 '13 at 4:15
  • Dell is sending me another controller. I will try to install OMSA in the meantime. It might be useful to have the log to determine if the new controller fixes the problem. However, I do remember having trouble installing OMSA in the past because I am running an unsupported OS (Mint). – jsp Nov 16 '13 at 16:24
  • 1
    If OMSA installation doesn't work out (I'd recommend trying both the newest version and maybe an older one like 5.2-ish) then you could also try exporting the log with MegaCLI. The tool isn't supported by Dell, but it's make by LSI, the OEM manufacturer of the PERC controllers. You could also boot from an OMSA Live disk/USB and create a DSET report from there. Just keep in mind that multiple reboots can flush out the old logs due to the limited space on the controller for logging history. – JimNim Nov 16 '13 at 22:36
0

You've got a couple of more sophisticated management options if you want to pull logs.

If you've got a UEFI Bios during boot, you can configure logging in there. This is described in the manual: ftp://ftp.dell.com/manuals/all-products/esuprt_ser_stor_net/esuprt_dell_adapters/poweredge-rc-h310_User%27s%20Guide_en-us.pdf

You can also install OpenManage and get the info that way: http://linux.dell.com/wiki/index.php/Repository/hardware

I'll third the suggestion on backups. Backup, backup, backup. In 2013 due to high areal density, buggy filesystems, and peculiarities with SSDs it is nigh-impossible to actually recover data from corrupted drives. Assume total data loss during any failure.

Even RAID 5 is considered risky nowadays. Or RAID at all. I've repeatedly run into problems with controller faults corrupting BOTH drives in a basic RAID 1 mirror array and variations (RAID 10). Same problems with RAID 5 and RAID 6.

It is now considered best practice to use RAID mainly for RAID 0 Striping and to use a completely independent system (separate drives/array) for backups using backup software.

In other words, you're generally considered to be better off backing up a single hard drive to an external USB hard drive than creating a mirror array.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.