VMware Host CPU Usage Climbing

Question

This is somewhat of a longshot, but I am wondering if anyone might be able to explain the following:

I have a VMware Host Server running Ubuntu Server 8.04 LTS, and VMware Server 2.0. The User CPU usage on the host keeps climbing, until I reboot the server when it goes back down to almost nothing - and starts to climb again. This is ever since I swapped a Server 2003 VM for a Server 2008 one (upgraded my DC, and migrated to 2008). I can't find any problem with the Windows server though, or any abnormally high CPU usage on the VM itself.

I have been keeping up-to-date on the host, so I've been through about 3 kernel upgrades, numerous recompiles of VMware, and a whole new VMware Server version when the last one was released not too long ago. I just can't figure this one out.

Any suggestions would be very well appreciated, I'm now just looking for things to try!

Answer 1

I can't answer the question, but I can add an extra little bit of anecdotal evidence. I have noticed this too, with a Linux hosts (Debian/Etch and Debian/Lenny).

After some mucking around when I first noticed the issue, I came to the conclusion that the issue is VMWare, not the guests themselves. Upon stopping all services in the VMs on a particular machine the CPU use remained high despite the OSs in the VMs doing nothing. On shutting down each of the four VMs the excessive host CPU use dropped by about 25% per VM (I didn't measure this by any scientific means, but certainly no one VM seems to be imposing most of the load). After restrting the VMs the CPU use remained where it used to be, even with the services in the VMs active, and the load has begun to slowly rise over time with no related rise in apparent useful activity.

In both the cases that I have noticed this happen, the host OS has been 32-bit Linux and the guest OSs have also been 32 bit Linux.

I have not seen it in all cases though. On my home server (64 bit Linux kernel with 32-bit user-land running one large and two small 32-bit Linux VMS and occasionally Windows VMS for testing) and the main dev/test host at work (64-bit Linux (both kernel and userland) host running mainly Windows VMs, some 32-bit and some 64) this aberrant behaviour doesn't seem to be present. All the above are running VMWare Server 2.

So to cut a long story short: it isn't just you, and it isn't just Windows based guests, but it doesn't seem to be a consistent problem (as many arrangements don't see it as do, in my limited experience). Though unfortunately I can't help any further than that as I've not had the time to look into the problem in more detail.

Answer 2

First, I've seen this with VMware Server, too - on both Windows and Linux. In my experience it was related to running both 64-bit and 32-bit VMs at the same time.

Though it may not be an option for you, I'd suggest ESXi - the lightweight, free edition of ESX.

Answer 3

I experience the same issue on a production server. Host OS is Debian Linux 64 bit. Guests are 4 Linux machines and 1 Windows XP. All guests are 32 bit. When i restart all VM services on the host, the CPU scheduling is fine and host CPU per VM is on par with real guest activity. However after some weeks the CPU scheduling increases until more or less the maximum available CPU time is consumed over the running VMs. The CPU usage on the host at that time is more or less 10x the initial CPU usage of the moment where the VMs just got started.

To me it seems a leak in the CPU scheduling on the host. Instaed of leaking RAM, it's leaking CPU cycles :-). How am I sure this doesnt happen on ESXi?

Thomasgg

Answer 4

I have a Host running Debian lenny, VMware workstation 7.1.

I tried VM of Win 2003 and winxp. The load is normal if I don't login. here is top:

top - 12:23:30 up 1 day, 45 min,  2 users,  load average: 0.65, 1.42, 1.02
Tasks: 144 total,   2 running, 142 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.4%us,  0.5%sy,  0.0%ni, 99.0%id,  0.0%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:   4149468k total,  3992488k used,   156980k free,   180692k buffers
Swap:  2731040k total,      708k used,  2730332k free,  3456976k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
10949 sixfoot   20   0  145m  57m  20m S    2  1.4   3:23.34 xulrunner-stub
13003 sixfoot   20   0 1705m 1.4g 1.4g S    1 36.4   0:25.50 vmware-vmx
 9376 sixfoot   20   0 95364  58m  37m S    0  1.4   2:34.04 vmware
    1 root      20   0  2104  688  588 S    0  0.0   0:01.70 init

But After I had loged in to windows VM for couple of minutes with doing nothing, the load of cpu climbed to 7 or 8. My host only have single cpu with 4 core.

top - 12:36:48 up 1 day, 58 min,  2 users,  load average: 6.18, 5.57, 3.16
Tasks: 144 total,   1 running, 143 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.2%sy,  0.0%ni, 26.7%id, 73.0%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:   4149468k total,  3950808k used,   198660k free,   186224k buffers
Swap:  2731040k total,      708k used,  2730332k free,  3410740k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
13003 sixfoot   20   0 1705m  93m  75m S    2  2.3   0:47.55 vmware-vmx
 2901 root      20   0 44848 6156 4084 S    1  0.1   1:12.56 Xorg
13379 root      20   0     0    0    0 D    1  0.0   0:00.06 pdflush
  884 root      15  -5     0    0    0 S    0  0.0   5:57.53 scsi_eh_0
 2779 xrdp      20   0 37504 9744  888 S    0  0.2   4:03.02 xrdp
 9376 sixfoot   20   0 95364  58m  37m S    0  1.4   2:37.41 vmware
    1 root      20   0  2104  688  588 S    0  0.0   0:01.70 init

In this situation, the host is very slow, and I can't even reboot.

I tried virtualbox after this. The load is 0 if I don't do anything in VM. The max load is less than 4.