On a VMware virtual machine which has severe performance problems I can see a constant average of 20+ percent CPU load for the TASKMGR.EXE (task manager) process. The apps running on this server have lower load, around 4 to 10 percent average. The VM is running Windows 2003 Server Standard with 3.75 GB assigned RAM. I suspect that the task manager CPU load has something to do with other VM instances on the VMWare server but could not see a similar value on internal ESXi systems (the problematic VM runs in the customers IT).
I had similar symptoms, not on a VM but a Win7 Laptop. Taskmgr.exe and other processes (like Winmerge.exe) were taking 13% of cpu (one out of 8). The issue was temporarily alleviated by restart, so I was doing that a lot.
After long time investigating it, I figured that that Webroot Secure Anywere (software for virus and other types of protection) WRSA.exe was the culprit. Once I terminated that process, everything started to be fast again like magic! Not sure yet what to use instead of Webroot for protection.
Well, what if you just quit the taskmanager? :-)
Every time I've seen ridiculous values for the task manager load it turned out to be swapping on the VM host due to memory shortage. Check your memory statistics on the host, especially the values of assigned and used memory and the swap used.
The performance on a virtualization server should be assessed first on the physical server, then on the VMs.
Inside the VM the time calculations are not accurate, especially if you over-comit the VCPUs.
Also the CPU time spend in system time could affect the performance very badly. Even a 10% could double the response time of a VM even if the CPU in total is less than 15% used.
Check if you are not over-commitning the memory and the host server is not swapping. This will flush your performance down to the drain. Make sure the VMware tools are installed and the paravirtualization (direct I/O, baloon) drivers are running.
Task Manager attempts to do things in real-time. It is polling resources at a set frequency. For any given set of metrics and load being monitored by task-manager, the number of instructions are per-determined at some fixed value by the algorithm at work.
If its using 20% CPU for a given set of metrics, its because there is not a lot of CPU to start with. If you run the same load on a machine with 10X the CPU, it will only take 2% of the CPU.
The % CPU used is therefore relative to the available CPU. In the case of a VM, it is relative to the physical capacity of the CPU its running on minus the capacity of the CPU consumed by the other VMs.
20% is only a lot relative to total CPU available to the VM its running on, and only important if you have better things to do with that CPU.
BTW - keep in mind that you should not worry about CPU being consumed unless it is being consumed and not doing something useful, or not doing enough because you have more stuff to do.
100% cpu consumption is not bad, it is desirable. It means that for a given workload, you are running as fast as you can. Monitor the CPU queue, if this is > 0, then you need more CPU.
Using < 100% CPU is not good, especially when you are trying to get work done. Keep in mind that CPU is a temporal resource. Unused cycles cannot be stored and have no value, they simply represent wasted opportunities to get things done.
An optimal system only shows spare CPU when its not loaded.
I had a sort of similar issue with a 2003 standard VM on vSphere 5.1. Initially I had configured a very simple VM (4GB vRAM, 20GB vDisk) with 1 socket with 2 cores. At idle, the VM CPU was constantly in the 30-50% range, even though there were no programs installed & it was fully up to date etc
I tried different CPU configurations and when I changed the # of vCores from 1 socket & 2 cores to 1 socket and 4 cores, (and restarted), everything calmed down - idling at 99%.
I don't know enough about vCPU methodology to offer an explanation but perhaps it will help someone
the host itself is a DL385 Gen8 with 2 x 16 core AMD's.