Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a LAMP system with CentOS 6.4, Apache 2.2.15, MySQL 5.5.32 and PHP 5.3.26 with APC 3.1.10 using a Drupal CMS (https) on a virtual machine with 4 CPUs and 6 GB RAM.

My problem is the CPU usage of the apache processes which will be up to 100% if I do a webpage crawler with 10 concurrent requests. What can I do to have less CPU usage? Any ideas? Here are my top result and config files with a mod_status server-status page:

top - 20:37:23 up 14 days, 22:52,  2 users,  load average: 8.54, 6.45, 3.22
Tasks: 180 total,   8 running, 171 sleeping,   0 stopped,   1 zombie
Cpu(s): 93.5%us,  6.2%sy,  0.0%ni,  0.0%id,  0.1%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:   8061508k total,  7634052k used,   427456k free,   146132k buffers
Swap:  8208376k total,    35116k used,  8173260k free,  5019896k cached
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                           
13155 apache    20   0  472m  95m  41m R 69.8  1.2   0:04.75 httpd                                                                                                                              
13144 apache    20   0  467m  88m  40m R 61.8  1.1   0:03.49 httpd                                                                                                                              
13141 apache    20   0  477m  97m  39m R 52.9  1.2   0:06.69 httpd                                                                                                                              
13162 apache    20   0  449m  71m  41m R 51.5  0.9   0:04.53 httpd                                                                                                                              
13145 apache    20   0  473m  93m  39m R 30.6  1.2   0:02.13 httpd                                                                                                                              
13136 apache    20   0  477m  98m  40m R 28.9  1.3   0:04.93 httpd                                                                                                                              
13158 apache    20   0  459m  79m  38m S 28.9  1.0   0:00.87 httpd                                                                                                                              
13160 apache    20   0  467m  84m  36m R 24.9  1.1   0:00.75 httpd                                                                                                                              
 3496 mysql     20   0 2454m 374m 5072 S 19.6  4.8  31:10.75 mysqld                                                                                                                             
13120 apache    20   0  460m  81m  40m S 15.3  1.0   0:05.63 httpd                                                                                                                              
13148 apache    20   0  462m  82m  39m R 12.0  1.1   0:01.79 httpd

I use this in my httpd.conf:

Timeout 60
KeepAlive On
MaxKeepAliveRequests 128
KeepAliveTimeout 10

<IfModule prefork.c>
  StartServers           16
  MinSpareServers        10
  MaxSpareServers        40
  ServerLimit       256
  MaxClients        256
  MaxRequestsPerChild  4000
</IfModule>

And this in my MySQL my.cnf (other is default):

max_allowed_packet = 64M
query_cache_size = 128M
query_cache_type = 1
query_cache_limit= 1M
slow-query-log-file=/var/log/mysqld/log-slow-queries.log // nothing logged -> no slow queries
table_cache = 1K
table_definition_cache = 4K
open_files_limit = 3K
thread_cache_size = 3

mod_statusshows me this if the server has nothing to do:

Parent Server Generation: 0
Server uptime: 23 seconds
Total accesses: 10 - Total Traffic: 49 kB
CPU Usage: u.04 s0 cu0 cs0 - .174% CPU load
.435 requests/sec - 2181 B/second - 5017 B/request
1 requests currently being processed, 15 idle workers
W_______________................................................
................................................................
................................................................
................................................................
Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process

Srv PID Acc M   CPU SS  Req Conn    Child   Slot    Client  VHost    Request
0-0 14223   10/10/10    W   0.04    0   0   49.1    0.05    0.05     IP www.test.com    GET /server-status HTTP/1.1

And with one website crawler (~10 concurrent requests):

Server uptime: 5 minutes 46 seconds
Total accesses: 1883 - Total Traffic: 65.5 MB
CPU Usage: u365.13 s17.79 cu0 cs0 - 111% CPU load
5.44 requests/sec - 193.8 kB/second - 35.6 kB/request
15 requests currently being processed, 10 idle workers
WWWW_WWKW___W_W__WW__K_WW.......................................
................................................................
................................................................
................................................................

    Srv     PID     Acc         M   CPU     SS  Req     Conn    Child   Slot    Client  VHost   Request
0-0     14223   90/207/207  W   2.07    0   0       902.0   1.61    1.61    IP  www.test.com    GET /server-status HTTP/1.1
1-0     14224   11/70/70    W   18.87   47  0       279.7   0.84    0.84    IP  www.test.com    GET /site1
2-0     14225   0/19/19     W   9.66    1   0       0.0     0.95    0.95    IP  www.test.com    GET /site2
3-0     14226   1/84/84     W   13.94   36  0       0.0     2.14    2.14    IP  www.test.com    GET /site3.pdf
4-0     14227   0/153/153   _   27.46   4   81      0.0     3.56    3.56    IP  www.test.com    GET /site4pdf
5-0     14228   12/78/78    W   17.59   1   0       275.8   2.67    2.67    IP  www.test.com    GET /site5.pdf
6-0     14229   10/90/90    W   14.62   0   0       611.4   3.85    3.85    IP  www.test.com    GET /site6
7-0     14230   4/53/53     K   34.12   0   141     9       81.9    2.48    IP  www.test.com    GET /site7
8-0     14231   1/114/114   W   12.39   27  0       0.0     2.29    2.29    IP  www.test.com    GET /site8.pdf
9-0     14232   0/106/106   _   9.68    2   64      0.0     6.33    6.33    IP  www.test.com    GET /site9
10-0    14233   0/39/39     _   15.66   5   1376    0.0     2.68    2.68    IP  www.test.com    GET /site10
11-0    14234   0/36/36     _   11.31   5   128     0.0     2.78    2.78    IP  www.test.com    GET /site11
12-0    14235   1/63/63     W   10.58   7   0       70.9    3.59    3.59    IP  www.test.com    GET /site12.pdf
13-0    14236   0/100/100   _   22.76   1   20      0.0     2.76    2.76    IP  www.test.com    GET /site13
14-0    14237   0/63/63     W   4.34    37  0       0.0     0.58    0.58    IP  www.test.com    GET /site14
15-0    14238   0/12/12     _   1.75    0   29      0.0     1.85    1.85    IP  www.test.com    GET /site15
16-0    14283   0/43/43     _   14.94   1   119     0.0     2.25    2.25    IP  www.test.com    GET /site16
17-0    14286   8/80/80     W   6.79    0   0       222.7   5.98    5.98    IP  www.test.com    GET /site17.pdf
18-0    14288   3/111/111   W   21.87   0   0       60.5    2.35    2.35    IP  www.test.com    GET /site18.pdf
19-0    14289   0/75/75     _   18.81   5   31      0.0     3.06    3.06    IP  www.test.com    GET /site19
20-0    14290   0/50/50     _   16.15   1   259     0.0     2.16    2.16    IP  www.test.com    GET /site20.pdf
21-0    14292   6/88/88     K   23.71   0   0       32.8    2.79    2.79    IP  www.test.com    GET /site21
22-0    14293   0/92/92     _   30.82   1   45      0.0     3.78    3.78    IP  www.test.com    GET /site22
23-0    14294   11/37/37    W   12.85   27  0       774.2   1.48    1.48    IP  www.test.com    GET /site23.pdf
24-0    14302   16/20/20    W   10.18   37  0       557.3   0.68    0.68    IP  www.test.com    GET /site24.pdf

At the end of the crawling I have up to Srv 46-0.

Swappping? This I did after the crawling has finished:

# swapon -s
Filename                Type        Size    Used    Priority
/dev/am-1                      partition         8208376    35172   -1
# free
         total       used       free     shared    buffers     cached
Mem:       8061508    7287776     773732          0     147220    4806004
-/+ buffers/cache:    2334552    5726956
Swap:      8208376      35172    8173204
share|improve this question
    
One option is to add varnish to sit in front of the drupal site, but this won't help in the crawler case if drupal tells varnish to not cache pages. And for it to help in the crawler case, you will have to have a long cache time. It may still hit issues of a crawler hitting an expired or non-cached page. However, if varnish works, it will reduce the CPU usage on the server. It will be good to limit the number of apache children, which will reduce the cpu contention, but will increase the time it takes for a child to get free, and respond to the next request. –  becomingwisest Jun 20 '13 at 21:13
    
But isn't it strange, that using Drupal I have to use Varnish or Squid to reduce CPU usage? So what is the problem with my LAMP configuration? Or isn't it possible to reduce the CPU usage in my case? –  Tim Jun 20 '13 at 21:22
    
Are you using mod_security? –  HTTP500 Jun 20 '13 at 22:13
    
No, my server-info does not list this module. –  Tim Jun 20 '13 at 22:18
add comment

2 Answers

Looking at some of this you seem to be swapping. Swapping is VERY BAD and causes large amounts of CPU to be consummed doing I/O swapping processes in and out. You might consider adding more memory....or reducing the number apache servers that startup.

share|improve this answer
    
Where do you see this? If I do a free I get, what you can see at my updated initial post. I think very very low swapping?! –  Tim Jun 20 '13 at 20:05
add comment

Use New Relic and look into the web transactions of your application.

New Relic now contains Drupal specific instrumentation as well (Modules, Hooks, Views).

Sorry if this sounds like a commercial but if you're running Drupal, it's really a no-brainer for the wealth of information it provides. I'm not affiliated with New Relic in anyway, just a happy customer.

If you want to do it the hard way profile your code (e.g. oprofile)

HTH

P.S.

You could bump up your thread_cache_size to a more reasonable number (e.g. 32) but I doubt it'll make a significant difference.

share|improve this answer
    
thread_cache_sizeto 32, no difference. I will take a look at this tool, but I still hope that it is a problem with my configuration. –  Tim Jun 20 '13 at 21:26
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.