I'd like to analyze how the processing time are distributed to individual CPU for a process ( KVM actually). In Linux a task can be migrated to other CPU during its life-cycle, the process may run in CPU0 for 1 sec then in CPU1 for 2 sec, but we can only get the overall processing time(3 sec) from /proc/pid/stat, but not 1sec and 2sec.

And we can get the per CPU aggregate processing time for all process in /proc/stat, but not for a particular process e.g. a KVM.

There are also some scheduling statics in /proc/pid/schedstat and /proc/schedstat, but I'm not clear how to derive it from statics like number of migrations.

This problem is also covered in the virt-top document, I think there is still no clear solution.

Do you have any ideas?


If we talk about processes, the Linux kernel has various and extensive ways to see what's going on under the hood. Unfortunately I don't know enough of this to give you a definitive answer, but I can suggest you to take a look at the BSD Process Accounting (CONFIG_BSD_PROCESS_ACCT), Export task/process statistics through netlink (CONFIG_TASKSTATS), Kernel Performance Events and Counters (CONFIG_PERF_EVENTS), OProfile (CONFIG_OPROFILE), GCOV based-kernel profiling (CONFIG_GCOV_KERNEL). Also if you're using CGroups for processes chances are that subsystem has some debug facility to offer too.

If you google for those facilities you'll easily come up with some articles that, together with their documentation and/or reference websites, can help you find which could help you and which couldn't.

I don't know KVM either, but I suppose it would show up as a kernel process so you can easily monitor it with the above systems. IIRC kvm also has the concept of back and front end drivers (maybe with other names?) so keep in mind part of a guest vm workload will show up as main host load.

One of those systems, or more than one combined, could help you achieving what you're after.

Last but not least, I couldn't imagine how that information could be useful to you - unless you're a kernel scheduler hacker but then you wouldn't be here asking on SF :) The scheduler should be doing a good enough job of preventing as many cpu hops as possible to keep caches warm (the scheduler knows about logical threads and multicore topology and uses that information) but there's usually a way to force some process to run on some CPUs only.

I'm sorry I wasn't able to reply with better precision but I hope this can help get you started.

| improve this answer | |
  • Thanks for loads of information. As for the intention of the question, I'm studying the vm migration/scheduling behavior of KVM, such statics is useful to design a better vm migration and placement strategy. Indeed, pin a vm to a CPU or assume the vm/process runs in one CPU is a partial solution, but I'm wonder if any better solution exists. – IZhen Dec 10 '11 at 10:54
  • I'd guess there's no such thing as a KVM migration or scheduling, afaik it is managed by the linux kernel scheduler. You should easily find out years of written articles and documentation about that. – Luke404 Dec 10 '11 at 21:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.