0

I have a server with 512MB RAM and 512MB swap file hosting a Wordpress multisite (8 websites). Not many visitors per day (about 50-100 per site per day).

Right now, everything is working fine, and my memory is like this:

$ free -m
             total       used       free     shared    buffers     cached
Mem:           484        300        183          0         12        102
-/+ buffers/cache:        185        298
Swap:          511         21        490

But about twice per day the sites are down and I cannot even ssh the server during several/many hours. Until something happens in the server and it kind of restarts by itself.

Analysing the logs, this is more or less what I get:

In /var/log/mysql/error.log

160105 13:00:59 InnoDB: The InnoDB memory heap is disabled
160105 13:01:03 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160105 13:01:03 InnoDB: Compressed tables use zlib 1.2.8
160105 13:01:03 InnoDB: Using Linux native AIO
160105 13:02:11 InnoDB: Initializing buffer pool, size = 128.0M
160105 16:05:18 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
160105 16:05:19 [Note] Plugin 'FEDERATED' is disabled.
160105 16:05:19 InnoDB: The InnoDB memory heap is disabled
160105 16:05:19 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160105 16:05:19 InnoDB: Compressed tables use zlib 1.2.8
160105 16:05:19 InnoDB: Using Linux native AIO
160105 16:05:19 InnoDB: Initializing buffer pool, size = 128.0M
160105 16:05:19 InnoDB: Completed initialization of buffer pool
160105 16:05:19 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
160105 16:05:19  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...

In var/log/apache2/error.log this is what I can see

[Mon Jan 04 13:10:12.538831 2016] [access_compat:error] [pid 18550] [client 178.170.253.10:61927] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:12:21.985186 2016] [access_compat:error] [pid 18550] [client 195.8.51.14:27348] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:12:44.698607 2016] [access_compat:error] [pid 20550] [client 176.124.168.83:52785] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:13:32.202567 2016] [access_compat:error] [pid 16906] [client 185.115.96.41:3342] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:13:52.323912 2016] [access_compat:error] [pid 19457] [client 176.51.88.244:4713] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:13:59.171594 2016] [access_compat:error] [pid 16906] [client 128.70.59.134:51542] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:15:01.406940 2016] [access_compat:error] [pid 19457] [client 178.136.213.198:57939] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:16:26.342924 2016] [access_compat:error] [pid 18550] [client 195.91.157.50:1994] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:17:40.552995 2016] [access_compat:error] [pid 22394] [client 84.18.126.78:6378] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:17:41.817137 2016] [access_compat:error] [pid 22393] [client 62.141.121.10:63297] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:18:18.672917 2016] [access_compat:error] [pid 22341] [client 178.140.252.61:62029] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:18:52.169060 2016] [access_compat:error] [pid 20550] [client 95.25.203.83:52487] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:19:27.421519 2016] [access_compat:error] [pid 22272] [client 95.32.218.30:2571] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 13:36:22.974166 2016] [evasive20:error] [pid 22341] [client 207.164.181.226:51490] client denied by server configuration: /var/www/images
[Mon Jan 04 13:36:22.996491 2016] [evasive20:error] [pid 22341] [client 207.164.181.226:51490] client denied by server configuration: /var/www/images
[Mon Jan 04 14:24:16.495532 2016] [access_compat:error] [pid 22343] [client 62.210.190.10:54167] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 14:24:16.542713 2016] [access_compat:error] [pid 22565] [client 62.210.190.10:62039] AH01797: client denied by server configuration: /var/www/wp-login.php
[Mon Jan 04 14:26:11.920278 2016] [:error] [pid 26081] [client 93.174.93.61:42308] WordPress database error Lost connection to MySQL server during query for query SELECT option_value FROM wp_options WHERE option_name = 'widget_calendar' LIMIT 1 made by include('wp-load.php'), require_once('wp-config.php'), require_once('wp-settings.php'), do_action('init'), call_user_func_array, wp_widgets_init, do_action('widgets_init'), call_user_func_array, WP_Widget_Factory->_register_widgets, WP_Widget->_register, WP_Widget->get_settings, get_option

I've protected the access to wp-login by IP (in .htaccess) to avoid undesirable access.

In /var/log/syslog I can see

Jan  5 13:00:56 ssc2 kernel: [445243.554498] Out of memory: Kill process 17543 (apache2) score 8 or sacrifice child
Jan  5 13:00:56 ssc2 kernel: [445243.576303] Killed process 17543 (apache2) total-vm:256148kB, anon-rss:2400kB, file-rss:392kB
Jan  5 13:00:56 ssc2 kernel: [445244.608497] apache2 invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0

I saw this repeated many times in /var/log/syslog as well:

Jan  5 12:48:48 ssc2 kernel: [444475.655836]  [<ffffffff81718e24>] dump_header+0x7f/0x1f1
Jan  5 12:48:48 ssc2 kernel: [444475.655841]  [<ffffffff81151ebe>] oom_kill_process+0x1ce/0x330
Jan  5 12:48:48 ssc2 kernel: [444475.655850]  [<ffffffff812d4815>] ? security_capable_noaudit+0x15/0x20
Jan  5 12:48:48 ssc2 kernel: [444475.655853]  [<ffffffff811525f4>] out_of_memory+0x414/0x450
Jan  5 12:48:48 ssc2 kernel: [444475.655857]  [<ffffffff8115896c>] __alloc_pages_nodemask+0xa5c/0xb80

I guess my Swap memory is getting full and the server is not responding, but in some cases it can last 7 hours!!!

Any help would be appreciated

1

Your server is exhausting the memory you've allocated to it. You will need to either reduce memory usage (ditch apache for nginx, move MySQL to a different host, etc) or add more RAM to this server.

  • The point is that the server was working fine for some months, and one day it started to give some memory issues. That was the reason to create the swap file. But now the server is down during several hours. My priority is to have the server always accesible and ssh for restarting it when needed. – nano Jan 5 '16 at 17:24
  • Right, of course. I've given you options for what you'll need to do to address the problem. – EEAA Jan 5 '16 at 17:25
  • Thanks! I've tried moving the DB to other server. It looks it doesn't crash any more, but the response time is pretty high. It takes several seconds to load each page. – nano Jan 6 '16 at 8:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.