I have a linux box hosting a pretty low-traffic site. It's an Amazon EC2 "small" instance, running Ubuntu 10.04. When I run "top" on it, I usually see a load of between 2 and 3, which in my experience is pretty high. However, not much seems to really be happening on the machine -- the CPU is almost always idle, and by watching the apache2 access.log file, I can see that not a lot of requests are coming through. How can I figure out what processes are waiting for the CPU, so that I can try to understand why the load metric is so high?
Using top you can see which threads are running and which ones are sleeping. This should allow you to at least know what is draining your resources if we're talking about a CPU bottleneck.
Should bring up a screen like this:
The column named S (that is the 8th one from the right) shows S for sleeping threads and R for running threads. You can choose the sorting order by pressing F (capital) and choosing the Process Status field. You can then reverse the sort order by pressing R (capital) so you can see the running threads first.
If your problem is not a CPU bottleneck I'll need some more info to assist you. Maybe you could post your top like I did on the above example.
Hope this helps.
If you want to know if you're experiencing some kind of I/O bottleneck you can issue the following command:
The load indicator is not caused only by high CPU usage. Processes can wait for other reasons such as disk and network I/O.
On Amazon EC2 small instances, disk I/O is very bad, especially /dev/sda2.
You can run iotop (apt-get install iotop on debian/ubuntu) to see which process consume I/O.