Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I don't know what or who is using this amount of CPU, the machine is a bi-quadcore with 16 GB ram. Running Ubuntu 10.04

Every core seems to be using 10-20 % except for one core that seems to be using 100% constantly.

At the moment I have about 14 Virtual Machines running on it, some have 4 cores available but most of them only 1.

In top I notice :

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

7873 root      20   0  530m  14m 3220 S  103  0.1   5597:41 kvm

I tried using kvm_stat :

kvm statistics

 efer_reload               2400       0
 exits               2199561167    6100
 fpu_reload          1244255128     136
 halt_exits          4368568581    1189
 halt_wakeup         1714293806     103
 host_state_reload   7549984578    1572
 hypercalls                   0       0
 insn_emulation      1398403526    3069
 insn_emulation_fail          3       0
 invlpg               767635659      37
 io_exits            2822598937     395
 irq_exits           1449081931     283
 irq_injections      4773146061    1220
 irq_window           125330658       8
 largepages                   0       0
 mmio_exits           326399432       0
 mmu_cache_miss       192709243       7
 mmu_flooded           69103717       4
 mmu_pde_zapped       259908526      15
 mmu_pte_updated            156       0
 mmu_pte_write        329155981      19
 mmu_recycled            856835       0
 mmu_shadow_zapped    192679259       5
 mmu_unsync               19380      -6
 nmi_injections               0       0
 nmi_window                   0       0
 pf_fixed            6517240715     389
 pf_guest            4434843050     217
 remote_tlb_flush     582797544      34
 request_irq                  0       0
 signal_exits                 5       0
 tlb_flush           1708903974     201

But I'll be honest I do not understand how to read this and how to know if there is a bottleneck somewhere. How can a process go into 103 % load ? Is this thread related ? Should I be worried or is this normal behaviour ?

share|improve this question

2 Answers 2

up vote 2 down vote accepted

If you press 'c' key inside top you can see the full command and understand which VM is (but a ps aux|grep 7873 will do it too).

After you discovered which VM is generating those CPU load, you should check it, like if it have some stuck process or it's hung (usually, 100% kvm process are VM stucks at the bootloader stage).

share|improve this answer

One possible explanation: There is one designated CPU is responsible for servicing the interrupt for the network interface. This designated CPU is responsible for ALL packets inbound for ALL Virtual Machines. That designated CPU then interrupts the CPU assigned to the VM for the packet's destination.

Intel's answer to this is SR-IOV. If you have very fast network interfaces and are receiving large numbers of packets, that may explain why your experiencing this effect.

Also you can look into tuning your network interrupt setting in Linux.

Good luck. I would like to hear back if this is helpful :)

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.