This morning we had an odd problem on a server running hyper-v. It was the second time we've seen the exact same problem int he last 6 months and I am a bit mystified as to the cause.
The physical host and the virtual machines cease having access to the network and the physical console of the host displays a black screen. Unfortunately I wasn't me who investigated but a user each time, so I can't confirm anything else. In each case the only solution was to power-cycle the server.
Up until here its just a driver problem. The box is up to date with the 8.70 support pack from HP.
After rebooting no network interfaces are available (the list shows up empty). In device manager the 4 interfaces show up with a little yellow triangle and the event viewer contains the following event
HP NC382i: Network controller failed to exchange interface with the bus driver.
Which makes no sense to me. Disabling and reenabling the devices had no effect.
The solution each time has been to uninstall the driver and let windows redetect them. After redetection you then have to reassociate the 'new' interfaces with the hyper-v networking config.
Its a win2008 R2Enterprise box running Hyper-v and 3 client machines. They've been installed and running for over a year now. Installed from scratch, not migrated.
The box is an HP DL385 G6. It has Broadcom NICs which are rebadged by HP to appear as NC382i controllers. The nic driver I have loaded is the hp one v188.8.131.52. The latest drivers appear to be 184.108.40.206 - so not particularly different.
Anyone seen anything similar or know what may be going on? I may schedule an upgrade to the NIC drivers but I don't think a minor update is likely to have much of an effect given the behaviour of the server.
I should also add that the host server was still logging events right up until it was rebooted - no anomalous events other than an odd duplicate netbios name which had been going on since the night before. No recent reboots, nor installations in the last couple of months. No new apps - this is a hyper-v server after all.
Update: I just realised that I hadn't updated this posting. I opened an incident with HP and we went over the drivers and firmware. Everything checked out ok so they suggested that we remove and reload the drivers and reflash the network card firmware. If the problem repeats itself they've said they will swap out the motherboard.
That was a couple of weeks ago and its been stable so far. This problem is very intermittent, so only time will tell. Have to give it a couple of months.