Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I've been trying to search over ther internet for a solution to fix this. I'm the new IT person for my organization and our previous IT have not kept any records on certain things. I do understand, that its a bad practice but I'm now making all these on a documentation for any future reference.

Having said that; recenty I came across an issue on our server. We're using an HP Proliant DL180 Gen6 server with ESXi 5.0 ... The issue is; that I'm unable to power up certain VM's as it gave me I/O error. Below seen was the error;

Reason: 0 (Input/output error). Cannot open the disk '/vmfs/volumes/4e7a4edb-08851e40-0c1e-1cc1de700f23/EON-GATEWAY (192.168.0.1 )/EON-GATEWAY ( 192.168.0.1 )-000001.vmdk' or one of the snapshot disks it depends on.

So to speak, I powered down all the VM's and restarted the host to jump into BIOS for an observation on the RAID. I do not know what type of RAID that the server is on as it shows something like;

Error on SLOT1 : bay 11 -- (as I remember)

Is there a way for me to check what exactly the issue is.. Because, I can see that the effected hard disk still flashes green color LED. Out of 12 bay.. bay 1 shows an orange color LED & bay 4 shows nothing at all.

I'm pretty much confused how to get this sorted. If anyone can guide me what exactly I need to do to get this sorted or may be a hint on how to check the RAID / array info.??

Update

Below seen images are from smart array controller...

enter image description here

enter image description here

enter image description here

Here's a video link to the server HDD's. I'm still curious as now the bay 1 flashes blue & amber while others bays are in blue (on the smart array screen as seen above)..

share|improve this question
    
When you select a disk in the array configuration utility, the blue UID indicator comes on so you can identify the disk. That's why you're seeing that. –  ewwhite Mar 27 at 15:34
    
Okay! so it showed blue UID indicator on all except for bay1 which keeps on indicating amber/blue UID continuously. Also, I've updated my answer once again with the error that I've got on screen... Could you advise me whether replacing the disk will loose any DATA(s) on a RAID 6 or usually will it recover as the status says recovering.\ –  AzkerM Mar 27 at 15:43
3  
No. Replace the failed disk. –  ewwhite Apr 1 at 8:28
1  
Download and run the HP Service Pack for ProLiant DVD .ISO. –  ewwhite Apr 1 at 11:01
1  
Just plug the disk in. You should really become familiar with how this equipment works. –  ewwhite Apr 7 at 7:43

4 Answers 4

up vote 6 down vote accepted

This could be a VMware issue or a locking problem on the virtual disk. Can you capture the full error message? Do other virtual machines power on without problems?

Despite that, it appears you have a physical storage issue, too.

Here's what the HP Smart Array P410 configuration output on a DL180 G6 looks like:

      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 2 TB, OK)
      physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 2 TB, OK)
      physicaldrive 1I:1:3 (port 1I:box 1:bay 3, SAS, 2 TB, OK)
      physicaldrive 1I:1:4 (port 1I:box 1:bay 4, SAS, 2 TB, OK)
      physicaldrive 2I:1:5 (port 2I:box 1:bay 5, SAS, 2 TB, OK)
      physicaldrive 2I:1:6 (port 2I:box 1:bay 6, SAS, 2 TB, OK)

Are you sure that you're not mistaking the drive designation of 1I:1:1, which means (port 1I:box 1:bay 1) for "SLOT1 : Bay 11"? That would explain the amber/orange light in the first drive bay.

Given that this server was not documented well, there's a high probability that it was also configured with RAID5 (mean? probably).

  • Does the server boot?
  • What error messages do you see at POST?
  • Do you have to press any keys on the keyboard to allow the system to boot? (e.g. F1)
  • What capacity and type of disks are installed in the server?

If the server is on, you can view the RAID configuration from within ESXi. Do this by navigating to: Hardware Status > Sensors > Storage.

If your ESXi was installed using an HP-specific VMware image, you will see the RAID configuration there.

enter image description here

If you don't see anything inside of VMware, you will need to reboot and view the RAID configuration at the BIOS level.

When the system is powered on, you want to hit the F8 key when prompted to enter the Smart Array P410 configuration utility.

Once inside, select "View Logical Drives".

enter image description here

This will show you the RAID health status and you can hit Enter for details. This will tell you conclusively which disks are good/bad/missing in the array.

share|improve this answer
    
Wow! your explanation pretty much makes sense to me.. Server does boot & the VM(s) also work perfectly except for few.. I'd say currently its just the gateway VM which I assume, stored on bay 11... Bay 1 shows an amber light, bay 4 shows nothing, bay11 shows green light in a slow flow but not with the rest of the bays.. Other bays flash green light in a same kind of sequence.. and I do not see storage under VM hardware status.. :( –  AzkerM Mar 27 at 13:59
    
@AzkerM Then you need to look at the BIOS configuration to see the RAID setup. –  ewwhite Mar 27 at 14:04
    
That's is what I'm planning to do now. All my worry is that it flashes the screen & go quickly... I cannot read the signs or the errors it shows properly.. Will check and post it back here. –  AzkerM Mar 27 at 14:06
    
Do I have to configure RAID monitoring under ESXi similar to this or is it an inbuilt feature? –  AzkerM Mar 27 at 14:15
2  
@AzkerM Yes. Replace the disk. –  ewwhite Mar 29 at 21:21

I may be wrong but I think you have two problems here.

Yes you appear to have a physical disk issue, if you can avoid the downtime then boot up off the HP SPP/ACU image and go into ACU, run the diagnostics and replace parts as needed.

The first error however suggests the datastore is IP-based, i.e. NFS or iSCSI, rather than a local SAS/SATA disk such as the ones you're having real problems with. Have you got other IP based datastores? If so I'd look at where they're based and see if something's been switched off or deleted.

share|improve this answer
    
It's worse than that. The VMs are on local storage, but the name in vCenter is literally EON-GATEWAY (192.168.0.1 ). –  ewwhite Mar 27 at 13:19

If you're lucky, your predecessor might have installed a proper HP image of the esxi server on the box, in which case you should be able to access the HP System Management Homepage remotely:

https:// ipofyourserver :2381

This should be able to tell you a little bit more about the general health of the server (which also includes the arrays).

If not, you should reboot the server and hit F8 after the P410i controller is done initializing. That will get you into the ORCA (Option Rom Configuration of Arrays). Select "Show logical drives". This should give you a list of the local logical drives, and will also say whether the array(s) are healthy or not. Note that you might have to "press any key" to actually see the P410i initialization messages, after the HP logo has appeared.

One last thing: I've seen on several occasions that something goes haywire in the internal workings of the storage box in the server, which will either render the LEDs in the drives mute (off), or scramble them so a healthy drive can be blinking amber instead of green. Just a fair warning, not to take the drive activity LEDs too serious :)

share|improve this answer

You might want to consider running a firmware upgrade dvd on the server as well. That controller firmware is ancient! I would urge you to download the firmware dvd, once you get the raid-5 array back in shape. There's considerable fixes and improvements in the later versions of the controller firmware.

Download it from here:

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/swdDetails/?sp4ts.oid=3884083&spf_p.tpst=swdMain&spf_p.prp_swdMain=wsrp-navigationalState%3Didx%253D%257CswItem%253DMTX_9ed665a89aba447d925937f38b%257CswEnvOID%253D4115%257CitemLocale%253D%257CswLang%253D%257Cmode%253D%257Caction%253DdriverDocument&javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.endCacheTok=com.vignette.cachetoken

You either put it on a usb drive and boot from it, burn it to a cd, or simply mount the iso file via ilo. Note that running the dvd will also upgrade the bios, the nic- and the ilo firmware.

share|improve this answer
    
Its a DL180 Gen6 server, and the link you gave me was DL380... Also, I'm on a RAID6 not 5 though both seems similar... :) –  AzkerM Mar 28 at 14:17
    
The firmware DVD is for all DL and ML servers. –  N-3 Apr 4 at 10:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.