0

Our server cpu load is high todays sometimes reaches to 560! .. We have the lastest Cpanel/whm and the kernel is update!while the load average is :

Load Averages: 39.05 75.01 45.33 the apache log is:

Current Time: Sunday, 30-Jan-2011 01:50:13 EST
Restart Time: Saturday, 29-Jan-2011 21:51:20 EST
Parent Server Generation: 2
Server uptime: 3 hours 58 minutes 53 seconds
Total accesses: 149493 - Total Traffic: 2.4 GB
CPU Usage: u9.17 s10.66 cu42.82 cs0 - .437% CPU load
10.4 requests/sec - 174.6 kB/second - 16.7 kB/request
121 requests currently being processed, 42 idle workers
W_WWW.__W_..W.W_W_WCWW..W...W.WWW.WWWW.WW.C_W_.W.WW.WC..W.WW._WW
.W_.W.W..__._WWWW.._.WW.CC.C.._W.WC.WW_WW._W....W.WWW.W.WWW.W..W
WW.....WW.W_WWWWW..WCRW..WWCW.WWW____.WWWWCW_W._._WW_W.._.W._..W
_W..W.._WW.W.._.._W..._WW.W._WWW..__W.WWW._WWW_....WW_._C._..W._
Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process

What cause this high cpu load while the apache cpu load is fine? the mysql process is also fine.. the cpu load is still high even if I stop mail-http-mysql services!

1

5 Answers 5

2
  1. See Apache access/error log file. Without logs, it is difficult to say something.
  2. If you have monitoring system(Cacti,Zabbix,...) see performance graphics.
  3. Install nginx frontend
  4. Check statistics(top,ps,iostat,vmstat,iotop)
  5. Read system log file(/var/log/kernel.log,/var/log/syslog)
0

If you have a raid setup you might check on the status. 3ware hardware raid provides nice command-line utilities. Software raid is managed with the 'mdadm' tool. I don't think installing nginx is a good suggestion for a cPanel server. More than likely this is a i/o bottleneck which just needs some fine tuning.

1
  • Contacting to Cpanel, they told me that the problem is because of the SOFT RAID! and the I/O wait time! .. unfortunately the DataCenter support is not going to solve the issue..,
    – kenby
    Feb 1, 2011 at 7:57
0

Looks like quite a small box to me is that correct? You may want to consider upgrading but solving the current issue would be even better!

As alvosu said, you should install nginx or similar (litespeed,lighttpd) to lighten the load a bit. Apache looks pretty busy there. I also notice that in just short of 4 hours you have almost 150,000 requests - 625 requests a minute or over 12 requests per second over the past 4 hours (Averaged). If you have a small box which from looking at this it looks like you do (Although I have very little information) Apache will be keeping that CPU very busy indeed.

Have you considered this may be the result of a denial of service attack?

3
  • Thank for your great response .. yes I think there should be DDOS Attack to the server, but the DC (Iweb) said that there is no DDOS to our server! ..
    – kenby
    Feb 1, 2011 at 7:55
  • 12 requests per second isn't that much, particularly if he has 150+ Apache workers. Not sure how you're determining that it's a small box based off the data provided.
    – ceejayoz
    Jun 4, 2011 at 1:45
  • Just a brief update on this, 12 requests per second can cause a high load dependant on what is being requested. For example, if the (D)DoS attack is calling a CPU intensive script such as a large database operation of even hitting the frontend of a forum this can cause some serious load issues.
    – Chris
    Nov 30, 2011 at 16:41
0

10.4 req/second is nothing really. You have a lot of connections 'sending' data; you might be better off examining the output from top, or even better:

watch -n 1 -d cat /proc/interrupts which will give you a better idea to know if you have an IO or network issue.

If you see one thread in top maxed out at wa, hi, or si then you should see if it would be pertinent to adjust the affinity for the interrupt responsible for the that behavior.

-2

I recommend you check out Cloudflare. It will sit in front of your website and act as a transparent proxy. Across the sites we use that are using it right now, we are seeing it caches about 75% of the http requests and 2/3rds of the bandwidth.

This lets your server focus on scripts and will reduce the disk i/o that is currently hurting you.

5
  • How do you know it's disk I/O hurting him?
    – ceejayoz
    Jun 4, 2011 at 1:44
  • here is what the questioner said in a comment: Contacting to Cpanel, they told me that the problem is because of the SOFT RAID! and the I/O wait time! .. unfortunately the DataCenter support is not going to solve the issue..
    – n8whnp
    Jun 4, 2011 at 18:23
  • What Cpanel says and what is actually happening may be entirely different. I've seen a lot of third-party vendors assign blame at random too many times to take that sort of statement as truth without doing my own benchmarking.
    – ceejayoz
    Jun 5, 2011 at 3:30
  • If the work load is cpu bound instead of IO bound, getting rid of 2/3rds of the requests will still be helpful. If they are getting ddosed, Cloudflare will block those attacks. If its apache overhead, cloudflare will reduce the work apache is doing. I have watched low end vps servers see their load go from 75-50 down to 3-4 with cloudflare caching. I don't work for CF or make any money off them. But as a server admin who supports websites for a living, its a very helpful project and I am sure it won't hurt his performance. How would adding CDN level static caching hurt this situation?
    – n8whnp
    Jun 5, 2011 at 5:12
  • I'd rather spend time diagnosing and fixing the underlying problem. 10-15 requests per second should be easily handleable by even a small Apache box. It certainly shouldn't spike load up to 560.
    – ceejayoz
    Jun 5, 2011 at 15:18

Not the answer you're looking for? Browse other questions tagged .