4

My site receives a lot of traffic for 2 hours during the day (2000 hits per minute). The rest of the day receives less traffic(500e hits per minute). I have been experimenting with the MaxClients and MaxSpareServers values but I still get some downtime during peek hours.

How can I calculate the best values for my configuration based on the amount of ram that I have ?

Each process is like 36-40 M of Memory

             total       used       free     shared    buffers     cached
Mem:          3096        793       2302          0          0          0
-/+ buffers/cache:        793       2302
Swap:  

      0          0          0

Values that I am using now 

<IfModule prefork.c>
StartServers 10
MinSpareServers 22
MaxSpareServers 60
ServerLimit 90
MaxClients 90
MaxRequestsPerChild  400

UPDATE: I found this from my hosting provider.

http://kb.mediatemple.net/questions/246/(dv)+HOWTO:+Basic+Apache+performance+tuning+(httpd)

I have the extreme package and I used 190 in MaxClients instead of 200 I tried it for half a night and I got some malloc error command line messages but using the free command and got like 800 M in the free column.

Improve this question

10 Answers 10

Reset to default
1

If your server suffers from peaks it may be as well because apache handle the process & threads creation very slowly (in mpm_prefork mode it's 1 new subprocess per second). If you know your 'high traffic' needs (let's say it's 200 process) then tell apache to get always thes 200 proces ready, even if they do nothing 90% of tim. You'll be ready when people comes in.

Check this handy presentation (old but cool) http://www.techpresentations.org/Hacking_Apache_HTTP_Server_at_Yahoo!

No need for StartServers, MaxSpareServers, MinSpareServers - just MaxClients (constant pool size)

Improve this answer
0

Answer depends greatly on the type of content you are serving. Is it dynamic or static content? It is a wide-range of content or a single document?

You can use memcached to store the documents in memory which will allow it to scale considerably (eliminating a disk bottleneck) but it all depends on how much RAM you have and if the content changes much.

Improve this answer
1
  • I am using memcached , but thanks
    – rbm
    Dec 20, 2010 at 16:10
0

Need to test depending on your server hardware (especially memory) and the efficiency of the site you are running. What is happening when you get downtime? Are you hitting these limits you have set? Are you running out of memory?

Improve this answer
0

You may also want to look at StartServers and ServerLimit.

ServerLimit sets the maximum number of servers, effectively limiting MaxClients.

StartServers sets the number of servers at startup. If, for instance, you're running a lottery or something where in those 2 hours everyone hits that site at once, it may be that it's unable to spawn children fast enough.

All that being said, it really depends on what your content is. If it's a big CGI script or something, then apache's got to deal with it. You may want to look at your machine setup (for reference, we used to run burningman.com with a few dozen sub-sites (bulletin boards, image gallery, etc) and mailing on it on a single dual-p3/900mhz with 1GB of RAM and only ever hit software limits.

Improve this answer
0

If you are in a tightly memory constrained situation (VPS) then you may need to lower the default maxclients to prevent swap death.

If you aren't, don't touch it.

Tuning the apache children is one of those things like custom compiling a kernel where people overwhelmingly do more harm than good, even if the harm is just wasting an afternoon for no benefit. It turns out the httpd project guys over at apache are pretty good at what they do! I mean who'd a thunk it?

Improve this answer
6
  • Indeed I am in a memory constrained situation. Maybe I need to get a bigger VPS. How to know how much RAM and CPU is needed to handle that traffic?
    – rbm
    Dec 20, 2010 at 17:30
  • Varies by stack, but a rough rule of thumb is to look at the RSS column in a "ps auxfw" output, take the average of your httpd processees and multiply it by the number of children for an overall apache memory footprint. So if you've got a 512mb system to work with and each of your children eats 25mb I'd keep it around 16 children max, to leave some room for the fs cache. If you're running other stuff on the box (mysql, memcache, cron jobs) then leave more room accordingly.
    – cagenut
    Dec 20, 2010 at 18:18
  • for cpu you need to know your average http request response times. just to pull an example number out of the air, lets say they're 200ms. That means you can service 5 http requests per second per cpu core. again, really rough starting point formula. if your processes spend a ton of time stuck in an iowait state for memcache or mysql then you can run more requests per second across the same cpu cores because they'll switch back and forth between the workers quickly.
    – cagenut
    Dec 20, 2010 at 18:27
  • The last test was with MaxClients 90 and the result of "free -m" was almost 700
    – rbm
    Dec 21, 2010 at 14:53
  • I updated the question with info about free memory and memory of each process.
    – rbm
    Dec 21, 2010 at 16:10
0

Is all the traffic for dynamic content? Have you considered using nginx in front of apache to serve all the static content? It would free all of apache's workers to deal with just the dynamic contet.

Improve this answer
1
  • I already moved all my static traffic to another server and I still have problems, Thanks anyway
    – rbm
    Dec 21, 2010 at 14:38
0

I agree with what cagenut said. Don't really touch it.

You could also consider using a http proxy like Varnish or Squid. But once again, you need some RAM for this one too.

Improve this answer
1
  • The result of "free -m" is like 700. The thing with reverse proxy is that I need to count the request individually(no caching allowed). I already offloaded all the static content to other servers.
    – rbm
    Dec 21, 2010 at 14:47
0

If you can do some benchmarking. Use can your siege, it will give you a rough idea on what your server can handle. Secondly, Apache is also CPU intensive. Even if you had Gigs of RAM and a weak CPU it wouldn't scale.

A optimization trick is to use a reverse proxy to front apache. nginx is a light weight webserver which functions very well as a reverse proxy.

Improve this answer
1
  • The last test was with MaxClients 90 and the result of "free -m" was almost 700. The thing with reverse proxy is that I need to count the request individually(no caching allowed). I already offloaded all the static content to other servers.
    – rbm
    Dec 21, 2010 at 14:51
0

You have to ensure memory for the OS, if you consume all memory available the system will hang. Usually, static content will use 2/5 MB, and dynamic 20/25 MB, you have 36/40 MB

So, let's do some maths:

Total Memory (in MB) / Total Memory per Process = MaxClients.

Supose you are going to reserve 512 MB for OS, you have 2584 MB for other purposes like apache.

2584 MB / 36 MB = 71.7
2584 MB / 40 MB = 64.4

ServerLimit limits MaxClients, use the same value, but do not use arbitrary numbers.
Use binary numbers, or try to approach to them based on your results. Example:

StartServers 5
MinSpareServers 5
MaxSpareServers 10
ServerLimit 64
MaxClients 64
MaxRequestsPerChild 1024

Also, do some apache's tunning:

KeepAlive On
MaxKeepAliveRequests 64
KeepAliveTimeout 2

I have 1 server with similar configuration, with 3 million monthly visits.

Improve this answer
2
  • I found a recommendation of my hosting provider that suggest to put 200 in the MaxClients. I updated the question to reflect that.
    – rbm
    Dec 22, 2010 at 17:37
  • Take it as a recommendation from your hosting provider, the point is that they really don't know how your site application works. Put your malloc error message here, and you will try to find out what's going on.
    – Matias Dominoni
    Jan 2, 2011 at 2:42
0

You could also try to use e.g. Nginx for static files and then need less Apache processes with their high memory footprint for the remaining requests.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .