1

I'm running a Virtual Machine server on Hyper-V that stalls about once a week, effectively rendering certain services it is running to be useless.

-The physical host machine

  • CPU: 6-core Xeon E5-2620 2.10GHz
  • RAM: 16GB
  • OS: Windows Server 2008 R2
  • Services
    • File Services
    • Hyper-V

-The Virtual Machine

  • Processor
    • Logical Processors: 4
    • Virtual Machine Limit: 100
    • Relative Weight: 100
  • Memory (Dynamic)
    • Memory Buffer: 20%
    • Memory Weight: High
  • Network Adapter: The only VM on this Virtual Network assigned to its own port
  • OS: Windows Server 2008 R2
  • Services
    • File Services
    • Print and Document Services
    • Web Server (IIS)
    • Windows Server Update Services

Here are the symptoms:

-When the server stalls I can...

  • Ping the server
  • Query the server using PowerShell CmdLets such as Get-Service

-When the server stalls I cannot...

  • Log into to the server remotely or through Hyper-V. I can send the Ctrl-Alt-Del command to the VM, but it hangs before making it to the login screen.
  • Connect to a network share hosted on the server
  • Print anything. The spooler gets backed up and all printing stops.

I setup a performance alert and data collectors to track the % Processor Time for each process and the total % Processor Time. During the stall period, the total % Processor Time averaged about 5%. I also checked the memory on the host machine, and there was always at least a few GBs available.

There are other VM servers running simultaneously on the same host. All other VMs seem to be unaffected during the stall period.

The VM server eventually starts functioning normally after a stall period of anywhere from 10-30 min. I could not find any relevant errors or warnings in the event logs. Does anyone have any idea why the server stalls in this way? Is there anything else I can check to narrow down the problem?

  • You need to monitor other resource groups in addition to CPU, like memory, disk and network. That should give you a better view of what might be causing the stalling – Zero Subnet Jul 7 '15 at 19:12
  • I setup data collectors for processes, memory, physical disk, network interface, and print queue. Hopefully, that will be enough data to narrow down the problem when it happens again. – Sherlock Bytes Jul 7 '15 at 22:57
  • @YodaPop Do you have any special drivers running on that server? I've seen it several times that faulty drivers (or hardware, but obviously not in your case)can halt the whole system and I couldn't see anything in the performance counters. – EliadTech Jul 8 '15 at 8:47
  • It happened again recently. None of the alerts I setup were triggered. I'm pretty sure the issue has nothing to do with a resource bottleneck. – Sherlock Bytes Jul 9 '15 at 16:05
  • 1
    Scratch what I said about the resource alerts not being triggered. The memory alert is being triggered. There was an average of 200MB available out of 9337MB, which is not good. – Sherlock Bytes Jul 9 '15 at 16:57
1

You can setup Performance Monitor to log memory used per process. Under available counters, scroll down to the Process object, expand it. Add Private Bytes, Working Set and Working Set-Private. Make sure to select so it would show you a breakdown per individual process instead of totals. That will show you the memory allocated per process and hopefully point you to the culprit.

Usually machines don't stall that bad when memory gets that low. But at least it's the one place you can start now.

1

We've had issues in the past when virtual machines have consumed too much memory and the host has locked up. There is a way to reserve some memory for Hyper-V which might help you. This solved some problems for us.

The registry path to use is explained here. It's a DWORD and should be specified in MB, for example 2048.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization\MemoryReserve

Reboot after configuring.

Also, you don't describe your storage environment. If your VMs and Host are sharing disks, that could give you problems.

  • This is a good suggestion and probably something I will take advantage of. However, I don't think it applies to my issue because there was memory available on the parent (host) when the problems occurred on only 1 out of the 3 VM servers running on the host. A related adjustment I would like to try is to change the dynamic memory maximum RAM setting for all of the VMs that are running, placing a reasonable limit on the amount of memory they can scale to. – Sherlock Bytes Jul 10 '15 at 21:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.