0

I have a lemp stack for wordpress 1gb server. Mysql keeps killing processes, one of which is netdata. I keep having to go into the ssh console to restart it as a result.

This is what i see in the error log :

Out of memory: Kill process 29383 (mysqld) score 244 or sacrifice child
Killed process 29383 (mysqld) total-vm:1169884kB, anon-rss:288068kB, file-
rss:0kB, shmem-rss:0kB

Do I need more memory? I'm currently getting an average of 70- 100 visitors per day for a food blog.

Nginx configuration

user www-data;
worker_processes 1;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 1024;
    use epoll;
    multi_accept on;
}

http {

##
# Basic Settings
##

sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
keepalive_requests 100000;
types_hash_max_size 2048;
server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;

#from ngninx optimize linode page
client_body_buffer_size      128k;
client_max_body_size         10m;
client_header_buffer_size    1k;
large_client_header_buffers  4 4k;
output_buffers               1 32k;
postpone_output              1460;
client_header_timeout  3m;
client_body_timeout    3m;
send_timeout           3m;
open_file_cache max=1000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 5;
open_file_cache_errors off;

include /etc/nginx/mime.types;
default_type application/octet-stream;

##
# SSL Settings
##
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;

##
# Logging Settings
##

access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;

##
# Gzip Settings
##

gzip on;
gzip_min_length 1000;
gzip_types text/html application/x-javascript text/css application/javascript text/javascript text/plain text/xml application/json application/vnd.ms-fontobject application/x-font-opentype application/x-font-truetype application/x-font-ttf application/xml font/eot font/opentype font/otf image/svg+xml image/vnd.microsoft.icon;
#gzip_disable "msie6";
gzip_disable "MSIE [1-6]\.";

# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

##
# Virtual Host Configs
##

include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}

MYSQL config

[mysqld]
max_allowed_packet = 1M
thread_stack = 128K
max_connections = 50
table_open_cache = 32M
key_buffer_size = 64M

top results mysqld Ver 5.7.19-0ubuntu0.17.04.1 for Linux on x86_64 ((Ubuntu)

KiB Mem : 82.9/1013064  
KiB Swap:100.0/262140  

PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
23755 www-data  20   0  445704  70616   9572 S 18.9  7.0   0:03.49 php-fpm7.0
11958 root      20   0  727764  10704   2036 S  3.0  1.1  46:25.45 fail2ban-se+
14667 www-data  20   0  145976   4768   2976 S  1.3  0.5   7:03.61 nginx
22808 mysql     20   0 1116544 204516   3360 S  0.7 20.2   0:14.66 mysqld
22868 netdata   20   0  179348  21292   1116 S  0.3  2.1   0:05.70 netdata
2

No you don't need more memory. I run five Wordpress sizes on an AWS ec2.nano, which has 512MB of RAM - half what you're using.

It's actually only using 363MB of RAM, the rest is cache. Two PHP processes are taking 37% of RAM, MySQL is taking 9%, Nginx is taking 1.3%.

             total       used       free     shared    buffers     cached
Mem:        501296     363152     138144     127004      37892     188776
-/+ buffers/cache:     136484     364812
Swap:       524284     119740     404544

The key for me was reducing the RAM MySQL used, particularly around the performance schema. I have a tutorial on MySQL on low memory systems here, which you can read, but the key parts are below.

You may also want to add virtual memory to your server. I did this on EC2 with EBS disk (ie a remote NAS) and it works just fine. Details here.

[mysqld]

# Disable performance schema to hugely reduce RAM usage
performance_schema = OFF

# I'm not sure if these two commands make any difference, but they don't seem to hurt
innodb_buffer_pool_size=30M
innodb_log_buffer_size=256K

[mysqld_safe]

# Settings to reduce RAM
innodb_buffer_pool_size=25M
innodb_log_buffer_size=256K
query_cache_size=10000
max_connections=30
key_buffer_size=80
innodb_ft_cache_size=1600000
innoinnodb_ft_total_cache_size=32000000
table_definition_cache=150

# Settings to reduce RAM: per thread or per operation settings
thread_stack=131072
sort_buffer_size=32K
read_buffer_size=8200
read_rnd_buffer_size=8200
max_heap_table_size=16K
tmp_table_size=50K
bulk_insert_buffer_size=100
join_buffer_size=128
net_buffer_length=1K
innodb_sort_buffer_size=64K

# Settings to reduce RAM: settings that relate to the binary log (if enabled)
binlog_cache_size=4K
binlog_stmt_cache_size=4K

Also, a note. MySQL isn't killing processes, the operating system kills processes because of the RAM MySQL is using. You could use a tool like Monit to restart processes, but you need to reduce RAM usage first.

The output of top, after hitting M to sort by memory usage, would've been useful information to include with your question. It would also have told you where to start as well.

Update

The problem could be PHP creating lots of processes that take RAM, which forces the OS to kill processes. The solution could be to limit PHP processes.

For me this is configured in /etc/php-fpm-5.6.d/www.conf but it could be different for you

pm.max_children = 4
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 2
pm.max_requests = 25
  • thank you tim, your first two lines errored out so I had to remove" # Disable performance schema to hugely reduce RAM usage performance_schema = OFF # I'm not sure if these two commands make any difference, but they don't seem to hurt innodb_buffer_pool_size=30M innodb_log_buffer_size=256K" i will give your suggestions a try and see if it makes a difference – patrickj Sep 4 '17 at 22:51
  • "performance_schema off" is the most critical part of the solution. It doesn't work for you please edit your post to include the MySQL version (mysqld -V) and the output of top after hitting M to sort by RAM usage. – Tim Sep 4 '17 at 22:59
  • i put the sql version and top out put above on the original post – patrickj Sep 4 '17 at 23:21
  • Ok, thanks. I formatted it so it's readable. Based on that top output memory usage is just fine, you have swap, I guess that PHP getting busy could be the cause of your problems - PHP takes heaps of RAM. Suggest you read my answer again, try it out, and if things don't work perfectly the first time you're going to need to think about things yourself and work out what's wrong. This worked on MySQL 5.6, but 5.7 might be slightly different. I'll edit my question soon with a suggestion regarding PHP, but the general idea is to limit PHP threads. You need to work out what's taking the RAM. – Tim Sep 5 '17 at 0:03
  • thank you for the help!! i will update if this works after monitoring for a few days – patrickj Sep 5 '17 at 0:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.