WHAT I'M TRYING TO DO
The server resource limits sometimes run tight; to help prevent memory exhaustion, I've had to limit server processes. I'm needing a little expert help to know if I'm on the right track, and perhaps spot any obvious settings changes that would help the system run more with more stability.
Recently my company upgraded to a VPS, from shared hosting. Basically we outgrew our shared hosting, and began to have problems due to the host suspending our site because of excessive CPU usage on the weekend. Our website users tend to double or triple on Friday and Saturday, every week, which is not unexpected in our case. (About 5000 visits [~2500 visitors] per day during the week, about 9500 visits [~4500 visitors] on weekends.)
Now that we are on a VPS, we have no CPU problems. (In fact, the CentOS WHM control panel says we are at ".000201% CPU load".) However, we are having out-of-memory problems, leading to crashes.
SUMMARY OF ISSUE
Our website is WordPress based. However, aside from comments, there is very little "write" activity; mostly users are simply seeing fairly static pages that we've created.
When we first upgraded to a VPS several months ago, in October 2012, the website ran well during the week, but choked on memory every weekend. Often it would crash repeatedly (5-20 times during a 24-hour period, sporadically), usually starting Friday evening, and continuing through Saturday afternoon.
During the week, the server ran consistently at 65-90% memory usage, and on the weekend it would hit 100%, causing crashes.
STEPS TAKEN TO CORRECT IT
Since I was new to VPS, I started with all the default settings. I later started tweaking, following advice I read about solving memory issues here on this site and other websites.
I've made adjustments to MySQL, PHP, and Apache, summarized below in "Current Configuration". I also recompiled Apache and PHP to remove unwanted modules. I installed a better caching plugin for WordPress (W3T), and added APC opcode caching. I also started using gz compression, and moved a lot of static files to a separate subdomain.
I wrote a nifty little script to check the server status on a schedule, and restart it as needed, and it also sends me a transcript of the server error log, to help troubleshoot. (I know, it's just a band-aid, if that. But it was important to keep the website online, since no one wants to sit around and monitor it on the weekend.)
Just recently, a week or so ago (January 2013), I upgraded the server RAM from 1 GB (2 GB burstable) to 2 GB (3 GB burstable). This seems to have fixed the majority of the problem, but still I get an occasional notice (once a week or so) that the server is hanging, along with "can't apply process slot" PHP errors.
It's an Apache server, running CentOS 6, Apache 2 (Worker MPM), PHP 5.3.20 (FastCGI/fcgi), and MySQL 5.5.28. 2 GB RAM (3 GB burstable), 24 CPUs.
MySQL currently uses about 618 MB, about 20.1% of RAM. PHP uses up to 89 MB per process. Apache uses up to 14 MB per process.
top - 15:31:13 up 89 days, 5:26, 1 user, load average: 1.54, 1.00, 0.70 Tasks: 49 total, 1 running, 48 sleeping, 0 stopped, 0 zombie Cpu(s): 0.2%us, 0.1%sy, 0.0%ni, 99.7%id, 0.1%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 3145728k total, 1046444k used, 2099284k free, 0k buffers Swap: 0k total, 0k used, 0k free, 0k cached
Unfortunately I do not have a current example of weekend/busiest time top output.
StartServers: 5 MinSpareThreads: 5 MaxSpareServers: 10 ServerLimit: 80 MaxClients: 56 MaxRequestsPerChild: 5000 KeepAlive: Off
MaxRequestsPerProcess 500 FcgidMaxProcesses 15 FcgidMinProcessesPerClass 0 FcgidMaxProcessesPerClass 8 FcgidIdleTimeout 30 FcgidIdleScanInterval 15 FcgidProcessLifeTime 60 FcgidIOTimeout 300 FcgidMaxRequestLen 268435456
[mysqld] max_user_connections = 75 net_buffer_length = 8K read_buffer_size = 256K read_rnd_buffer_size = 512K skip-external-locking sort_buffer_size = 512K # MyISAM # key_buffer_size = 32M myisam_sort_buffer_size = 16M #myisam_recover = FORCE,BACKUP # SAFETY # max_allowed_packet = 8M #max_connect_errors = 1000000 # CACHES AND LIMITS # tmp_table_size = 104M max_heap_table_size = 104M join_buffer_size = 208K #query_cache_type = 0 query_cache_size = 32M max_connections = 150 thread_cache_size = 4 #open_files_limit = 65535 table_cache = 512 #table_definition_cache = 1024 table_open_cache = 2048 wait_timeout = 300 # INNODB # #innodb_flush_method = O_DIRECT #innodb_log_files_in_group = 2 #innodb_log_file_size = 64M #innodb_flush_log_at_trx_commit = 1 #innodb_file_per_table = 1 innodb_buffer_pool_size = 416M # This setting ensures that aio limits are not exceeded # (default is 65536, each instance of mysql takes 2661 with this enabled) innodb_use_native_aio = 0 # LOGGING # log-slow-queries log-queries-not-using-indexes
Any help/suggestions would be much appreciated. The website address is 3abn.org.