Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a new HP ProLiant DL360 G7 system that is exhibiting a difficult-to-reproduce issue. The server randomly hangs at the "Power and Thermal Calibration in Progress..." screen during the POST process. This typically follows a warm-boot/reboot from the installed operating system.

enter image description here

The system stalls indefinitely at this point. Issuing a reset or cold-start via the ILO 3 power controls makes the system boot normally without incident.

When the system is in this state, the ILO 3 interface is fully accessible and all system health indicators are fine (all green). The server is in a climate-controlled data center with power connections to PDU. Ambient temperature is 64°F/17°C. The system was placed in a 24-hour component testing loop prior to deployment with no failures.

The primary operating system for this server is VMWare ESXi 5. We initially tried 5.0 and later a 5.1 build. Both were deployed via PXE boot and kickstart. In addition, we are testing with baremetal Windows and Red Hat Linux installations.

HP ProLiant systems have a comprehensive set of BIOS options. We've tried the default settings in addition to the Static high-performance profile. I've disabled the boot splash screen and just get a blinking cursor at that point versus the screenshot above. We've also tried some VMWare "best-practices" for BIOS config. We've seen an advisory from HP that seems to outline a similar issue, but did not fix our specific problem.

Suspecting a hardware issue, I had the vendor send an identical system for same-day delivery. The new server was a fully-identical build with the exception of disks. We moved the disks from the old server to the new. We experienced the same random booting issue on the replacement hardware.

I now have both servers running in parallel. The issue hits randomly on warm-boots. Cold boots don't seem to have the problem. I am looking into some of the more esoteric BIOS settings like disabling Turbo Boost or disabling the power calibration function entirely. I could try these, but they should not be necessary.

Any thoughts?

--edit--

System details:

  • DL360 G7 - 2 x X5670 Hex-Core CPU's
  • 96GB of RAM (12 x 8GB Low-Voltage DIMMs)
  • 2 x 146GB 15k SAS Hard Drives
  • 2 x 750W redundant power supplies

All firmware up-to-date as of latest HP Service Pack for ProLiant DVD release.

Calling HP and trawling the interwebz, I've seen mentions of a bad ILO 3 interaction, but this happens with the server on a physical console, too. HP also suggested power source, but this is in a data center rack that successfully powers other production systems.

Is there any chance that this could be a poor interaction between low-voltage DIMMs and the 750W power supplies? This server should be a supported configuration.

share|improve this question
1  
Any way to eliminate the disks as a possible cause? Any chance you can test with some alternate SAS or SATA disks? –  ErnieTheGeek Jan 10 '13 at 19:16
    
Yes, tested with a known-good set of disks in the second system. They're running in parallel. –  ewwhite Jan 10 '13 at 19:20
1  
The only time I've ever seen this was in a system (also a DL360 G7) where I was trying to use a non-HP card to provide storage. When I had both the SmartArray card and this other one in there, it did that. When I took either out, it passed. This is not your problem, but I pass on what I ran into. –  sysadmin1138 Jan 10 '13 at 19:36
1  
Possibly something network related? Try to duplicate without being connected to the network. –  ErnieTheGeek Jan 10 '13 at 19:43
1  
@TheCleaner Disabling Dynamic Power Capping is not an option on G7 servers. It was introduced for the Gen8 ProLiant series. –  ewwhite Jan 11 '13 at 16:24

1 Answer 1

up vote 34 down vote accepted

So, after bringing a third system into the mix, and experiencing the same issue, we began to question the environment. I dug up a copy of the HP ProLiant Servers Troubleshooting Guide and found the POST problems flowchart shown below.

enter image description here

Carefully running through the steps in the chart, we realized that the one constant across all of the servers was a KVM switch attached to the data center crash cart. This was a consumer-class USB-enabled KVM. As per the highlighted node in the flowchart, Do you have known good KVM?, I could not answer conclusively.

So, we unplugged the servers from the KVM switch and ran an automated boot, sleep 300; reboot sequence in rc.local. The servers had no issues with this, regardless of the normal DIMM, low-voltage DIMMs, PSU wattage, etc.

This was all the result of a poor interaction with a USB KVM switch. By virtue that this was the console, it ensured we'd see the failure if we were looking for it. Self-fulfilling...

share|improve this answer
1  
Wow, that's a good one! Glad you sussed this out. –  nedm Jan 11 '13 at 22:53
4  
Holy crow. +1 to question and answer. Good work; I probably would have overlooked that. "Known good" ? Of course it's known good - it's working, ain't it? –  mfinni Jan 14 '13 at 16:11
    
Thank You Very Much!!! it was definitely the KVM. Just disconnect the Video and plug the Monitor directly and the server runs smoothly again. After the O.S. load up I plugged the KVM back. I Think the problem was caused when I accidently touched the cables in the back of the server. The System halted and only react to this advise. –  user175039 May 23 '13 at 17:13
    
Any idea how a KVM would cause this? –  TheLQ Oct 3 '13 at 13:23
    
@TheLQ A cheap consumer-level KVM device was the cause here. There may have also been a problem with the keyboard. –  ewwhite Oct 10 '13 at 23:06

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.