I am running a Website that gets heavy usage at a particular time. I run apache 2.4 on an EC2 Ubuntu 16.04 server with 4 Core CPU and 16GB of RAM and 2GB Swap. Size of the average apache process is 35MB (approx). I have set up the KeepAlive value as ON and KeepAliveTimeout to a lower value of 5.
I am using MPM prefork and the configuration looks like:
<IfModule mpm_prefork_module> StartServers 5 MinSpareServers 5 MaxSpareServers 10 ServerLimit 350 MaxRequestWorkers 350 MaxConnectionsPerChild 20000 </IfModule>
When the number of concurrent users increases, htop shows that all the 4 cores are 100% utilized, however, the RAM still shows only 1.5 or 2GB utilized, out of the available 15GB.
Why isn't more memory being utilized? Is it normal to have this kind of CPU usage? If the number of users increases from here, the server starts to return an HTTP 522 error.
I found a single apache process taking up approximately 35MB of memory. I run a nodejs (for socketio) under PM2. MySQL is run from an RDS and is not on this server. Though, I have reduced some values as per the suggested calculations:
Available Memory - 14000MB
(leaving the remaining RAM for other purposes)
Apache Process usage - 40MB MaxRequestWorkers = 14000/40 = 350 StartServers 30% of MaxRequestWorkers
(should be 105, but I feel it is too high. Should I make it 105?)
MinSpareServers 5% of MaxRequestWorkers
(should be 17)
MaxSpareServers 10% of MaxClients
(Is it okay to have this value same as StartServers and as high as 105?)
ServerLimit = MaxRequestWorkers
Edit I modified the prefork configuration, using the suggested calculations:
<IfModule mpm_prefork_module> StartServers 105 MinSpareServers 17 MaxSpareServers 105 ServerLimit 350 MaxRequestWorkers 350 MaxConnectionsPerChild 20000 </IfModule>
I ran a script that makes 2 API calls, imitating a user coming on board and doing some action 4 times. Running 20 of these scripts parallel, with new configurations still gave a high load average value: