1

This morning we had an odd problem on a server running hyper-v. It was the second time we've seen the exact same problem int he last 6 months and I am a bit mystified as to the cause.

The physical host and the virtual machines cease having access to the network and the physical console of the host displays a black screen. Unfortunately I wasn't me who investigated but a user each time, so I can't confirm anything else. In each case the only solution was to power-cycle the server.

Up until here its just a driver problem. The box is up to date with the 8.70 support pack from HP.

After rebooting no network interfaces are available (the list shows up empty). In device manager the 4 interfaces show up with a little yellow triangle and the event viewer contains the following event

HP NC382i: Network controller failed to exchange interface with the bus driver.

Which makes no sense to me. Disabling and reenabling the devices had no effect.

The solution each time has been to uninstall the driver and let windows redetect them. After redetection you then have to reassociate the 'new' interfaces with the hyper-v networking config.

Its a win2008 R2Enterprise box running Hyper-v and 3 client machines. They've been installed and running for over a year now. Installed from scratch, not migrated.

The box is an HP DL385 G6. It has Broadcom NICs which are rebadged by HP to appear as NC382i controllers. The nic driver I have loaded is the hp one v6.2.9.0. The latest drivers appear to be 6.2.16.0 - so not particularly different.

Anyone seen anything similar or know what may be going on? I may schedule an upgrade to the NIC drivers but I don't think a minor update is likely to have much of an effect given the behaviour of the server.

I should also add that the host server was still logging events right up until it was rebooted - no anomalous events other than an odd duplicate netbios name which had been going on since the night before. No recent reboots, nor installations in the last couple of months. No new apps - this is a hyper-v server after all.

Update: I just realised that I hadn't updated this posting. I opened an incident with HP and we went over the drivers and firmware. Everything checked out ok so they suggested that we remove and reload the drivers and reflash the network card firmware. If the problem repeats itself they've said they will swap out the motherboard.

That was a couple of weeks ago and its been stable so far. This problem is very intermittent, so only time will tell. Have to give it a couple of months.

6
  • Also, doesn't support pack download updates direct from HP if you let it? This has been my understanding. If you run a recent version it will suggest the latest updates on the day you run it and download anything it needs. I have another more recent server on which I have the same model of NIC but which has the later drivers - something which I found odd when I noticed (after making my post above)
    – Ian Murphy
    Oct 14, 2011 at 22:01
  • I've just checked again and 8.70 is the latest and it says it was released 5 April 2011. This server is just a little over a year old and has been updated since we installed it.
    – Ian Murphy
    Oct 14, 2011 at 22:10
  • 1
    HP SUM 8.70 is definitely from 22 Jan 2010. They're not very hardware specific. When running SUM you can select the option to download the latest CPs from the FTP site, but this option is not checked by default. Looking at the NC382x cards, driver package 6.2.8.0 is the newest version (from HP or Broadcom); you must be looking at the NDIS driver 6.2.9.0 (what displays in Device Manager). Sounds like you might have the current driver. The 6.2.16.0 driver is for the 10GbE NC532x cards. Are you running HP NCU 10.40? What firmware are you running?
    – Chris S
    Oct 15, 2011 at 1:12
  • Sys management page seems to indicate that you've worked out whats going on. According to it we're running nc-series broadcom 6.2.8.0, so yes, up to date and, yes, I had been looking in the device manager. The NC382i firmware is 5.2.2. The page says 'HP NC-Series Broadcom Online Firmware Upgrade Utility' v2.1.5.9 (B), so I would guess that refers to the version of the utility and not the firmware it applies. NCU appears to be out of date - 10.0.0.0 - but 10.40 says its not valid for installation on this system. Odd. The Server isn't using teaming.
    – Ian Murphy
    Oct 16, 2011 at 21:23
  • I spoke to HP yesterday and after confirming that I had the server up to date, they suggested reinstalling the driver and reflashing the card bios. If that doesn't work, they will change the motherboard.
    – Ian Murphy
    Oct 18, 2011 at 12:54

0

You must log in to answer this question.

Browse other questions tagged .