Only one CPU load on multi core server

Question

top - 01:50:56 up 15 days, 30 min, 4 users, load average: 0.89, 1.17, 1.11 Tasks: 170 total, 1 running, 169 sleeping, 0 stopped, 0 zombie

Cpu0  : 56.6%us,  5.7%sy,  0.0%ni, 36.8%id,  0.0%wa,  0.0%hi,  0.9%si,  0.0%st
Cpu1  :  0.9%us,  0.0%sy,  0.0%ni, 99.1%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  8.3%us,  0.9%sy,  0.0%ni, 90.8%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  1.0%us,  1.0%sy,  0.0%ni, 98.1%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :  9.3%us,  1.2%sy,  0.0%ni, 88.4%id,  0.0%wa,  0.0%hi,  1.2%si,  0.0%st
Cpu5  :  0.9%us,  0.0%sy,  0.0%ni, 99.1%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  3.7%us,  0.9%sy,  0.9%ni, 94.4%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st

Mem: 8180768k total, 7169736k used, 1011032k free, 756132k buffers Swap: 2104472k total, 24k used, 2104448k free, 5849080k cached

19033 quinroot 20 0 384m 45m 19m S 5 0.6 0:31.52 httpd
19292 quinroot 20 0 375m 35m 18m R 5 0.4 0:15.10 httpd
19474 quinroot 20 0 368m 26m 16m S 5 0.3 0:04.26 httpd
19344 quinroot 20 0 373m 34m 18m S 5 0.4 0:13.30 httpd
19351 quinroot 20 0 373m 33m 18m S 5 0.4 0:12.82 httpd
19455 quinroot 20 0 369m 28m 17m S 5 0.4 0:06.20 httpd
19476 quinroot 20 0 369m 26m 15m R 5 0.3 0:04.48 httpd
19478 quinroot 20 0 368m 25m 14m S 5 0.3 0:04.40 httpd
19540 quinroot 20 0 367m 24m 15m S 5 0.3 0:01.28 httpd
19542 quinroot 20 0 367m 24m 15m S 5 0.3 0:01.26 httpd
19173 quinroot 20 0 379m 41m 19m S 4 0.5 0:24.40 httpd
19290 quinroot 20 0 375m 34m 18m S 4 0.4 0:15.28 httpd
19349 quinroot 20 0 373m 31m 16m S 4 0.4 0:12.86 httpd
19454 quinroot 20 0 369m 29m 17m S 4 0.4 0:06.14 httpd
19475 quinroot 20 0 368m 26m 15m S 4 0.3 0:04.10 httpd
19544 quinroot 20 0 367m 23m 14m S 4 0.3 0:01.28 httpd
19099 quinroot 20 0 380m 42m 20m S 3 0.5 0:25.90 httpd
19293 quinroot 20 0 374m 34m 17m R 3 0.4 0:15.24 httpd
19537 quinroot 20 0 367m 23m 14m S 3 0.3 0:01.08 httpd
19545 quinroot 20 0 367m 23m 14m S 3 0.3 0:01.16 httpd
19526 root 20 0 16940 1348 948 R 1 0.0 0:00.16 top
1 root 20 0 1064 392 324 S 0 0.0 0:07.62 init
2 root 15 -5 0 0 0 S 0 0.0 0:00.00 kthreadd

Why Apache always use only one CPU?

Answer 1

Looking at the load averages, I suspect the webserver isn't busy enough (i.e. serving enough concurrent requests) to require heavy use of multiple CPUs.

As you have many mostly idle httpd processes, you're likely using MPM prefork and have MinSpareServers set around 20. Having a number of idle httpd processes is normal, these are used to help Apache quickly respond, when traffic to your webserver increases.

The listing above of instantaneous CPU usage at some time isn't very indicative of long term usage (e.g. it might be an unlucky sample). However, as to why CPU #0 may be favoured:

• Apache MPM prework worker processes with lower PIDs tend to do more work (as confirmed by this comment). For example, the two processes you list with the highest TIME have the lowest process IDs (19033, 19099).
• Linux favours keeping processes on the same CPU, where possible (for cache performance): See man taskset ("Note that the Linux scheduler also supports natural CPU affinity: the scheduler attempts to keep processes on the same CPU as long as practical for performance reasons."). Since picking a random CPU is sub optimal (consider power usage), repeat use of CPU #0 seems sensible.

So I'd say your webserver is mostly being served by a couple of processes, and Linux is favouring running them on the same CPU for performance reasons.

If you'd like more information on Apache processes, enable the ExtendedStatus directive (uncomment the appropriate section in httpd.conf): This enables a status webpage which shows among other things, a list of processes and what they're currently serving.

hope this helps! :)

Lockie