How do I log CPU usage per process?

Question

I have a box on Linode that's going through weird behavior. Every now and then CPU and disk I/O will shoot to 100% and the server becomes unresponsive and has to be booted. I'd like to investigate better what's going on, but I don't know how to find who's responsible for all that CPU and I/O. I'm running Gentoo 2.6.18.

Answer 1

You could try to do something like this:

while true; do ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10 >> logfile.txt; echo "\n" >> logfile.txt; sleep 3; done

that would show you the top ten processes in terms of CPU usage. You can change the number of processes shown by changing the 10 in "head -10" to a different number, and how often it updates by changing the 3 in "sleep 3" or taking out the "sleep 3" part entirely.

Answer 2

Check out atop it will write a binary log of pretty much everything you would possibly want and then you can use a top like gui to go through the time slices of the day (default is to take the data every 5 minutes). http://www.atcomputing.nl/Tools/atop/

Answer 3

I think that munin is one of the goods tools of monitoring that will help your to get some information about your box's activities. Also, there are some command line tool like sar, iostat, ps, top for such use.

Answer 4

The other answers have only shown you how you can look at what's currently going on, which doesn't help if the system has been rebooted.

If you want this information recorded for posterity (or billing, or whatever other use you might also have), what you want is process accounting.

Here's a HOWTO I found, but I'll be honest -- it's been a decade since I've used process accounting.

http://tldp.org/HOWTO/Process-Accounting/

Answer 5

Doesn't Gentoo have the "top" command as wel?

machine:~/# top

should give you the running stats of which programs causes the most load.