Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm having problems with the load on my website. It's an amazon ec2 server with 15Gb ram and 4 CPUs behind an LB.

apachetop says I'm getting around 80 reqs per second which seems really low for this kind of server and the load ( given by top ) is usually around 15 but does increase to about 150 in 24 hrs. I'm seeing about 100 active apache processes at any time. Apache is in prefork mode.

Mysql is used very little on the server and there are almost no static files.

Here are my Apache settings:

Timeout 20
KeepAlive Off
MaxKeepAliveRequests 0
KeepAliveTimeout 3

<IfModule mpm_prefork_module>
    StartServers         40
    MinSpareServers      25
    MaxSpareServers      40
    ServerLimit         400
    MaxClients          400
    MaxRequestsPerChild  4
</IfModule>

Can anyone advise on how to tweak the settings? Thanx!

Edit: The config was gotten by trial and error. Any and I mean by a number, change to these lines make the load skyrocket in like 5 minutes. It literally jumps to like 200-300 in a matter of minutes. Especially MaxRequestsPerChild. I've tried with 10, 15, 100, 1000 and the load just skyrockets.

About php - there are actually only a few php files which aren't really that expensive at all. They just spit some simple stuff out.

If I turn on KeepAlive load also goes to space..

Edit: This is what server status gives:

Current Time: Monday, 16-Apr-2012 07:26:07 UTC
Restart Time: Monday, 16-Apr-2012 07:06:44 UTC
Parent Server Generation: 3
Server uptime: 19 minutes 22 seconds
Total accesses: 65803 - Total Traffic: 51.8 MB
CPU Usage: u24.25 s1.99 cu0 cs0 - 2.26% CPU load
56.6 requests/sec - 45.7 kB/second - 825 B/request
65 requests currently being processed, 0 idle workers

WWWCWWCCWWWWCCCWWCWWCWCWWCCWWWCCWWWCWWCWWWSWC..C..W...WCCCC.C.GW
CWCC.CWWCCC.WC..................................................
................................................................
................................................................
................................................................
................................................................
................

I can see a lot of connections in "Wait" mode but I'm not sure why the req/sec is so low. Only 56 and so many slots are empty...

Update: I have changed the code a little, removed all routines which could slow things down and I'm still getting only 100 reqs per second. Load is stabile at about 10...

I think these kind of machines should be able to do much more...

share|improve this question
    
did you see my last reply? Is this still an issue for you? –  Andre Lackmann Apr 19 '12 at 13:33

2 Answers 2

up vote 2 down vote accepted

Your situation suggests to me the PHP script is waiting on something to complete (like a SQL query or API call). You should be able to turn on apache's Extended Status module to get a much better idea of what apache is doing.

To do so, in your httpd.conf you'll need to enable mod_status and have 'ExtendedStatus On' (see: http://httpd.apache.org/docs/2.0/mod/mod_status.html#extendedstatus)

eg.

LoadModule status_module modules/mod_status.so
ExtendedStatus On

<Location /server-status>
 SetHandler server-status
 Order deny,allow
 Deny from all
 Allow from xxx.xxx.xxx.xxx
</Location>

Replace xxx.xxx.xxx.xxx with the IP(s) you want to be able to access the status page and then hit http://yoursite.com/server-status to view the output. Most of the configs are probably already in your .conf files or have been commented out.

Once you have access to it, you'll see something similar to:

K___K_W_........................................................
................................................................
................................................................
................................................................

Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" KeepAlive (read), "D" DNS Lookup, "L" Logging,
"G" Gracefully finishing, "." Open slot with no current process

This gives you a lot more idea of what is happening behind the scenes in Apache. If my first assumption is correct (ie. your PHP script is waiting on something), then you'll see lots of tasks in 'W' mode.

Give it a try - if you still need help, let me know what the above output looks like.

share|improve this answer
    
Thanx!!! You were right... most connections are in W mode... I updated the question with server-status response –  Mathew Apr 16 '12 at 7:30
1  
Thx for adding the server-status output. So, in the extended list underneath that output, it will show you which script is running for each child process. Are all/most of them the same script? If so, what is that script doing that is taking so long? This looks more and more like a slow script that is just taking a long time to complete due to an external dependency (disk, api, db or similar) –  Andre Lackmann Apr 16 '12 at 10:20
    
Problem identified - a php native function which loads it's ini file for every request was causing the problem. Our solution was to rewrite the method and take another approach while we didn't try to place the ini file in ram. That might have helped also. If I had started profiling sooner ( when this comment came, I would have fixed it sooner ). That's why this comment is accepted. –  Mathew Oct 17 '12 at 14:18

Even the load average of 15 is too high - it's indicative that the system is running into a bottleneck.

What's running in Apache?

If it's dynamic content, check if the CPUs are completely utilized; the code may need some optimization or you may need to scale to more EC2 instances to handle the CPU requirements of the content.

If it involves the clients loading a decent amount of data, then you may be overloading disk (check the iowait numbers in top) or even your available bandwidth.

Depending on what's running, you may be able to use an MPM with better performance than prefork.

MaxRequestsPerChild 4 is incredibly low - do you know why this is set? And why is KeepAlive Off set? Both of those settings are very bad from a performance perspective; can you give some background on how the config got to where it is?

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.