Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

help me please with my apache problem. I got an droplet at DigitalOcean with 512mb of RAM (Debian). There are 3 sites on MODX CMS (works on php). Today morning as i think system discovered an out of memory problem and started to kill processes (There was about 43 of apache procesess). It killed mysqld and started kills of apache2 one by one. Can you please point me which config i have to tune? Tuning-primer.sh for mysql tells that there is no problems with memory allocation.

Here is my my.cnf:

key_buffer = 16M 
max_allowed_packet = 16M 
thread_stack = 192K 
thread_cache_size = 8
 myisam-recover = BACKUP 
max_connections = 70 
max_heap_table_size = 25M 
tmp_table_size = 25M
 query_cache_limit = 1M 
query_cache_size = 25M 
[mysqldump] 
quick quote-names 
max_allowed_packet = 16M 
[isamchk] 
key_buffer = 16M

Here the logs:

Dec 10 06:41:27 cloud kernel: [1711384.052842] apache2 invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Dec 10 06:41:27 cloud kernel: [1711384.053081] apache2 cpuset=/ mems_allowed=0
Dec 10 06:41:27 cloud kernel: [1711384.053177] Pid: 21594, comm: apache2 Not tainted 3.2.0-4-amd64 #1 Debian 3.2.41-2+deb7u2
Dec 10 06:41:27 cloud kernel: [1711384.053362] Call Trace:
Dec 10 06:41:27 cloud kernel: [1711384.053453]  [<ffffffff810b6ac2>] ? dump_header+0x78/0x1bd
Dec 10 06:41:27 cloud kernel: [1711384.053580]  [<ffffffff81163cae>] ? security_real_capable_noaudit+0x40/0x4f
Dec 10 06:41:27 cloud kernel: [1711384.053727]  [<ffffffff810b6e8b>] ? oom_kill_process+0x49/0x271
Dec 10 06:41:27 cloud kernel: [1711384.053864]  [<ffffffff810b7586>] ? out_of_memory+0x2ea/0x337
Dec 10 06:41:27 cloud kernel: [1711384.054043]  [<ffffffff810bb214>] ? __alloc_pages_nodemask+0x629/0x7aa
Dec 10 06:41:27 cloud kernel: [1711384.054194]  [<ffffffff810e470a>] ? alloc_pages_current+0xc7/0xe4
Dec 10 06:41:27 cloud kernel: [1711384.054326]  [<ffffffff810b61c7>] ? filemap_fault+0x24f/0x33e
Dec 10 06:41:27 cloud kernel: [1711384.054454]  [<ffffffff810ce344>] ? __do_fault+0xc8/0x3ac
Dec 10 06:41:27 cloud kernel: [1711384.054572]  [<ffffffff810d08fb>] ? handle_pte_fault+0x298/0x79f
Dec 10 06:41:27 cloud kernel: [1711384.054700]  [<ffffffff810cde99>] ? pte_offset_kernel+0x16/0x35
Dec 10 06:41:27 cloud kernel: [1711384.054840]  [<ffffffff813509bd>] ? do_page_fault+0x312/0x337
Dec 10 06:41:27 cloud kernel: [1711384.054965]  [<ffffffff810d5ce0>] ? mmap_region+0x353/0x44a
Dec 10 06:41:27 cloud kernel: [1711384.055086]  [<ffffffff8134dfe5>] ? async_page_fault+0x25/0x30
Dec 10 06:41:27 cloud kernel: [1711384.055210] Mem-Info:
Dec 10 06:41:27 cloud kernel: [1711384.055265] Node 0 DMA per-cpu:
Dec 10 06:41:27 cloud kernel: [1711384.055346] CPU    0: hi:    0, btch:   1 usd:   0
Dec 10 06:41:27 cloud kernel: [1711384.055591] Node 0 DMA32 per-cpu:
Dec 10 06:41:27 cloud kernel: [1711384.055811] CPU    0: hi:  186, btch:  31 usd:   0
Dec 10 06:41:27 cloud kernel: [1711384.056074] active_anon:113121 inactive_anon:458 isolated_anon:0
Dec 10 06:41:27 cloud kernel: [1711384.056075]  active_file:92 inactive_file:104 isolated_file:45
Dec 10 06:41:27 cloud kernel: [1711384.056076]  unevictable:0 dirty:0 writeback:1 unstable:0
Dec 10 06:41:27 cloud kernel: [1711384.056077]  free:1210 slab_reclaimable:1522 slab_unreclaimable:2999
Dec 10 06:41:27 cloud kernel: [1711384.056078]  mapped:419 shmem:374 pagetables:5928 bounce:0
Dec 10 06:41:27 cloud kernel: [1711384.057383] Node 0 DMA free:2048kB min:84kB low:104kB high:124kB active_anon:13372kB inactive_anon:8kB active_file:0kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15680kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:44kB slab_unreclaimable:136kB kernel_stack:120kB pagetables:96kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:902 all_unreclaimable? yes
Dec 10 06:41:27 cloud kernel: [1711384.059096] lowmem_reserve[]: 0 489 489 489
Dec 10 06:41:27 cloud kernel: [1711384.059359] Node 0 DMA32 free:2792kB min:2784kB low:3480kB high:4176kB active_anon:439112kB inactive_anon:1824kB active_file:368kB inactive_file:392kB unevictable:0kB isolated(anon):0kB isolated(file):180kB present:500948kB mlocked:0kB dirty:0kB writeback:4kB mapped:1676kB shmem:1496kB slab_reclaimable:6044kB slab_unreclaimable:11860kB kernel_stack:1320kB pagetables:23616kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:59064 all_unreclaimable? yes
Dec 10 06:41:27 cloud kernel: [1711384.065944] lowmem_reserve[]: 0 0 0 0
Dec 10 06:41:27 cloud kernel: [1711384.066199] Node 0 DMA: 12*4kB 0*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2048kB
Dec 10 06:41:27 cloud kernel: [1711384.066781] Node 0 DMA32: 676*4kB 14*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2816kB
Dec 10 06:41:27 cloud kernel: [1711384.067365] 626 total pagecache pages
Dec 10 06:41:27 cloud kernel: [1711384.067585] 0 pages in swap cache
Dec 10 06:41:27 cloud kernel: [1711384.067797] Swap cache stats: add 0, delete 0, find 0/0
Dec 10 06:41:27 cloud kernel: [1711384.068060] Free swap  = 0kB
Dec 10 06:41:27 cloud kernel: [1711384.068271] Total swap = 0kB
Dec 10 06:41:27 cloud kernel: [1711384.069956] 131053 pages RAM
Dec 10 06:41:27 cloud kernel: [1711384.070168] 3819 pages reserved
Dec 10 06:41:27 cloud kernel: [1711384.070379] 77878 pages shared
Dec 10 06:41:27 cloud kernel: [1711384.070682] 122871 pages non-shared
Dec 10 06:41:27 cloud kernel: [1711384.070899] [ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
Dec 10 06:41:27 cloud kernel: [1711384.071343] [  311]     0   311     5355      157   0     -17         -1000 udevd
Dec 10 06:41:27 cloud kernel: [1711384.071782] [  449]     0   449     5330      139   0     -17         -1000 udevd
Dec 10 06:41:27 cloud kernel: [1711384.072250] [  450]     0   450     5361      156   0     -17         -1000 udevd
Dec 10 06:41:27 cloud kernel: [1711384.072695] [ 1866]     0  1866    13514      431   0       0             0 rsyslogd
Dec 10 06:41:27 cloud kernel: [1711384.073146] [ 1918]     0  1918     1028       35   0       0             0 acpid
Dec 10 06:41:27 cloud kernel: [1711384.073593] [ 1973]     0  1973    70397     2566   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.074044] [ 2000]     0  2000     4167       38   0       0             0 atd
Dec 10 06:41:27 cloud kernel: [1711384.074483] [ 2048]     0  2048     5101       63   0       0             0 cron
Dec 10 06:41:27 cloud kernel: [1711384.074921] [ 2078]   102  2078     7450       85   0       0             0 dbus-daemon
Dec 10 06:41:27 cloud kernel: [1711384.075370] [ 2487]     0  2487    12462      153   0     -17         -1000 sshd
Dec 10 06:41:27 cloud kernel: [1711384.075805] [ 2857]   101  2857    11182      108   0       0             0 exim4
Dec 10 06:41:27 cloud kernel: [1711384.076262] [ 2886]     0  2886     4063       39   0       0             0 getty
Dec 10 06:41:27 cloud kernel: [1711384.076720] [ 2887]     0  2887     4063       42   0       0             0 getty
Dec 10 06:41:27 cloud kernel: [1711384.077161] [ 2888]     0  2888     4063       40   0       0             0 getty
Dec 10 06:41:27 cloud kernel: [1711384.077603] [ 2889]     0  2889     4063       41   0       0             0 getty
Dec 10 06:41:27 cloud kernel: [1711384.078047] [ 2890]     0  2890     4063       41   0       0             0 getty
Dec 10 06:41:27 cloud kernel: [1711384.078536] [ 2891]     0  2891     4063       40   0       0             0 getty
Dec 10 06:41:27 cloud kernel: [1711384.078979] [ 3711]     0  3711    31922      278   0       0             0 console-kit-dae
Dec 10 06:41:27 cloud kernel: [1711384.079444] [21957]     0 21957     1044       37   0       0             0 mysqld_safe
Dec 10 06:41:27 cloud kernel: [1711384.079895] [22320]   105 22320    96631    19582   0       0             0 mysqld
Dec 10 06:41:27 cloud kernel: [1711384.080357] [22321]     0 22321     1396       26   0       0             0 logger
Dec 10 06:41:27 cloud kernel: [1711384.080835] [ 3086]    33  3086    41182     1994   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.081286] [21327]    33 21327    73497     5283   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.081731] [21404]    33 21404    73951     5828   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.082230] [21405]    33 21405    74016     5890   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.082675] [21419]    33 21419    73379     5166   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.083117] [21422]    33 21422    73312     5159   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.083558] [21526]    33 21526    73371     5187   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.083979] [21527]    33 21527    73433     5238   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.084425] [21528]    33 21528    73568     5335   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.084854] [21529]    33 21529    73184     5059   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.085285] [21547]    33 21547    73439     5220   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.085733] [21577]    33 21577    73459     5228   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.086223] [21578]    33 21578    73219     4983   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.086650] [21579]    33 21579    73464     5237   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.087091] [21580]    33 21580    73349     5082   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.087559] [21581]    33 21581    73349     5077   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.088043] [21582]    33 21582    73349     5100   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.088520] [21583]    33 21583    72404     4272   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.088972] [21584]    33 21584    72404     4272   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.089398] [21585]    33 21585    73349     5106   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.089821] [21586]    33 21586    73412     5192   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.090283] [21587]    33 21587    73242     5024   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.090748] [21588]    33 21588    73242     5023   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.091176] [21589]    33 21589    72652     4444   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.091602] [21590]    33 21590    71976     3758   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.092042] [21591]    33 21591    71381     3164   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.092501] [21592]    33 21592    70915     2766   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.092923] [21593]    33 21593    61758     2411   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.093349] [21594]    33 21594    70639     2497   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.093771] [21595]    33 21595    70639     2491   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.094235] [21596]    33 21596    70637     2490   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.094665] [21597]    33 21597    70628     2482   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.095096] [21598]    33 21598    70628     2483   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.095522] [21599]    33 21599    70628     2483   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.095951] [21600]    33 21600    70628     2482   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.096389] [21601]    33 21601    70628     2482   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.096827] [21602]    33 21602    70628     2482   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.097259] [21603]    33 21603    70628     2482   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.097684] [21604]    33 21604    70628     2482   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.098149] [21605]    33 21605    70628     2482   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.098575] [21606]    33 21606    70628     2482   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.099030] [21607]    33 21607    70628     2482   0       0             0 apache2
Dec 10 06:41:27 cloud kernel: [1711384.099451] Out of memory: Kill process 22320 (mysqld) score 154 or sacrifice child
Dec 10 06:41:27 cloud kernel: [1711384.099883] Killed process 22320 (mysqld) total-vm:386524kB, anon-rss:78328kB, file-rss:0kB
Dec 10 06:41:27 cloud mysqld_safe: Number of processes running now: 0
Dec 10 06:41:27 cloud mysqld_safe: mysqld restarted
Dec 10 06:41:28 cloud kernel: [1711385.712243] apache2 invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Dec 10 06:41:28 cloud kernel: [1711385.712810] apache2 cpuset=/ mems_allowed=0
Dec 10 06:41:28 cloud kernel: [1711385.713046] Pid: 21595, comm: apache2 Not tainted 3.2.0-4-amd64 #1 Debian 3.2.41-2+deb7u2
Dec 10 06:41:28 cloud kernel: [1711385.718027] Mem-Info:
Dec 10 06:41:28 cloud kernel: [1711385.718222] Node 0 DMA per-cpu:
Dec 10 06:41:28 cloud kernel: [1711385.718440] CPU    0: hi:    0, btch:   1 usd:   0
Dec 10 06:41:28 cloud kernel: [1711385.718683] Node 0 DMA32 per-cpu:
Dec 10 06:41:28 cloud kernel: [1711385.718903] CPU    0: hi:  186, btch:  31 usd:   0
Dec 10 06:41:28 cloud kernel: [1711385.719148] active_anon:113844 inactive_anon:461 isolated_anon:2
Dec 10 06:41:28 cloud kernel: [1711385.719149]  active_file:14 inactive_file:32 isolated_file:33
Dec 10 06:41:28 cloud kernel: [1711385.719150]  unevictable:0 dirty:0 writeback:20 unstable:0
Dec 10 06:41:28 cloud kernel: [1711385.719151]  free:1210 slab_reclaimable:1148 slab_unreclaimable:2950
Dec 10 06:41:28 cloud kernel: [1711385.719152]  mapped:351 shmem:373 pagetables:5843 bounce:0
Dec 10 06:41:28 cloud kernel: [1711385.725329] Node 0 DMA free:2048kB min:84kB low:104kB high:124kB active_anon:13388kB inactive_anon:8kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15680kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:32kB slab_unreclaimable:136kB kernel_stack:120kB pagetables:108kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Dec 10 06:41:28 cloud kernel: [1711385.726946] lowmem_reserve[]: 0 489 489 489
Dec 10 06:41:28 cloud kernel: [1711385.727208] Node 0 DMA32 free:2792kB min:2784kB low:3480kB high:4176kB active_anon:441988kB inactive_anon:1836kB active_file:56kB inactive_file:128kB unevictable:0kB isolated(anon):8kB isolated(file):132kB present:500948kB mlocked:0kB dirty:0kB writeback:80kB mapped:1404kB shmem:1492kB slab_reclaimable:4560kB slab_unreclaimable:11664kB kernel_stack:1144kB pagetables:23264kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:33394 all_unreclaimable? yes
Dec 10 06:41:28 cloud kernel: [1711385.728895] lowmem_reserve[]: 0 0 0 0
Dec 10 06:41:28 cloud kernel: [1711385.729147] Node 0 DMA: 14*4kB 0*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2056kB
Dec 10 06:41:28 cloud kernel: [1711385.729730] Node 0 DMA32: 689*4kB 5*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2796kB
Dec 10 06:41:28 cloud kernel: [1711385.730347] 455 total pagecache pages
Dec 10 06:41:28 cloud kernel: [1711385.730567] 0 pages in swap cache
Dec 10 06:41:28 cloud kernel: [1711385.730782] Swap cache stats: add 0, delete 0, find 0/0
Dec 10 06:41:28 cloud kernel: [1711385.731031] Free swap  = 0kB
Dec 10 06:41:28 cloud kernel: [1711385.731234] Total swap = 0kB
Dec 10 06:41:28 cloud kernel: [1711385.732988] 131053 pages RAM
Dec 10 06:41:28 cloud kernel: [1711385.733203] 3819 pages reserved
Dec 10 06:41:28 cloud kernel: [1711385.733412] 71552 pages shared
Dec 10 06:41:28 cloud kernel: [1711385.733649] 123160 pages non-shared
Dec 10 06:41:28 cloud kernel: [1711385.733952] [ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
Dec 10 06:41:28 cloud kernel: [1711385.734422] [  311]     0   311     5355      157   0     -17         -1000 udevd
Dec 10 06:41:28 cloud kernel: [1711385.734866] [  449]     0   449     5330      139   0     -17         -1000 udevd
Dec 10 06:41:28 cloud kernel: [1711385.735305] [  450]     0   450     5361      156   0     -17         -1000 udevd
Dec 10 06:41:28 cloud kernel: [1711385.735750] [ 1866]     0  1866    13514      484   0       0             0 rsyslogd
Dec 10 06:41:28 cloud kernel: [1711385.736210] [ 1918]     0  1918     1028       35   0       0             0 acpid
Dec 10 06:41:28 cloud kernel: [1711385.736648] [ 1973]     0  1973    70397     2538   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.737091] [ 2000]     0  2000     4167       38   0       0             0 atd
Dec 10 06:41:28 cloud kernel: [1711385.737524] [ 2048]     0  2048     5101       63   0       0             0 cron
Dec 10 06:41:28 cloud kernel: [1711385.737964] [ 2078]   102  2078     7450       85   0       0             0 dbus-daemon
Dec 10 06:41:28 cloud kernel: [1711385.738413] [ 2487]     0  2487    12462      153   0     -17         -1000 sshd
Dec 10 06:41:28 cloud kernel: [1711385.738853] [ 2857]   101  2857    11182      108   0       0             0 exim4
Dec 10 06:41:28 cloud kernel: [1711385.739291] [ 2886]     0  2886     4063       39   0       0             0 getty
Dec 10 06:41:28 cloud kernel: [1711385.739736] [ 2887]     0  2887     4063       42   0       0             0 getty
Dec 10 06:41:28 cloud kernel: [1711385.740186] [ 2888]     0  2888     4063       40   0       0             0 getty
Dec 10 06:41:28 cloud kernel: [1711385.740634] [ 2889]     0  2889     4063       41   0       0             0 getty
Dec 10 06:41:28 cloud kernel: [1711385.741388] [ 2890]     0  2890     4063       41   0       0             0 getty
Dec 10 06:41:28 cloud kernel: [1711385.741828] [ 2891]     0  2891     4063       40   0       0             0 getty
Dec 10 06:41:28 cloud kernel: [1711385.742276] [ 3711]     0  3711    31922      278   0       0             0 console-kit-dae
Dec 10 06:41:28 cloud kernel: [1711385.742736] [21957]     0 21957     1044       37   0       0             0 mysqld_safe
Dec 10 06:41:28 cloud kernel: [1711385.743188] [ 3086]    33  3086    41182     1994   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.743627] [21327]    33 21327    73497     5280   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.744164] [21404]    33 21404    73951     5825   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.744607] [21405]    33 21405    74016     5890   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.745209] [21419]    33 21419    73379     5163   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.745721] [21422]    33 21422    73312     5159   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.746433] [21526]    33 21526    73371     5187   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.746880] [21527]    33 21527    73433     5235   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.747322] [21528]    33 21528    73568     5332   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.747765] [21529]    33 21529    73212     5059   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.748247] [21547]    33 21547    73439     5255   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.748691] [21577]    33 21577    73459     5240   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.749159] [21578]    33 21578    73219     4991   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.749602] [21579]    33 21579    73464     5245   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.750072] [21580]    33 21580    73349     5082   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.750515] [21581]    33 21581    73349     5077   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.750953] [21582]    33 21582    73349     5150   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.751393] [21583]    33 21583    72404     4272   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.751836] [21584]    33 21584    72404     4272   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.752294] [21585]    33 21585    73349     5106   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.752747] [21586]    33 21586    73412     5196   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.753210] [21587]    33 21587    73250     5042   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.753653] [21588]    33 21588    73250     5047   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.754100] [21589]    33 21589    73107     4943   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.754543] [21590]    33 21590    72544     4343   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.754980] [21591]    33 21591    72109     3947   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.755426] [21592]    33 21592    72109     3892   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.755868] [21594]    33 21594    71852     3674   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.756324] [21595]    33 21595    71929     3743   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.756771] [21596]    33 21596    71929     3742   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.757210] [21597]    33 21597    71763     3568   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.757652] [21598]    33 21598    71929     3743   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.758098] [21599]    33 21599    71929     3742   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.758541] [21600]    33 21600    71763     3570   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.758979] [21601]    33 21601    71763     3568   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.759430] [21602]    33 21602    71575     3406   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.759868] [21603]    33 21603    71123     2948   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.760327] [21604]    33 21604    71087     2857   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.760766] [21605]    33 21605    71087     2859   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.761210] [21606]    33 21606    71575     3406   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.761656] [21607]    33 21607    71087     2857   0       0             0 apache2
Dec 10 06:41:28 cloud kernel: [1711385.762101] [21620]   105 21620    30993      668   0       0             0 mysqld
Dec 10 06:41:28 cloud kernel: [1711385.762542] [21621]     0 21621     1396       23   0       0             0 logger
Dec 10 06:41:28 cloud kernel: [1711385.762978] Out of memory: Kill process 21405 (apache2) score 47 or sacrifice child
Dec 10 06:41:28 cloud kernel: [1711385.763420] Killed process 21405 (apache2) total-vm:296064kB, anon-rss:23560kB, file-rss:0kB
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 [Note] Plugin 'FEDERATED' is disabled.
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 InnoDB: The InnoDB memory heap is disabled
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 InnoDB: Mutexes and rw_locks use GCC atomic builtins
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 InnoDB: Compressed tables use zlib 1.2.7
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 InnoDB: Using Linux native AIO
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 InnoDB: Initializing buffer pool, size = 128.0M
Dec 10 06:41:29 cloud mysqld: InnoDB: mmap(137363456 bytes) failed; errno 12
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 InnoDB: Completed initialization of buffer pool
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 InnoDB: Fatal error: cannot allocate memory for the buffer pool
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 [ERROR] Plugin 'InnoDB' init function returned error.
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 [ERROR] Unknown/unsupported storage engine: InnoDB
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 [ERROR] Aborting
Dec 10 06:41:29 cloud mysqld: 
Dec 10 06:41:29 cloud mysqld: 131210  6:41:28 [Note] /usr/sbin/mysqld: Shutdown complete
Dec 10 06:41:29 cloud mysqld: 
Dec 10 06:41:30 cloud mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
share|improve this question
    
Can you post your MySQL config? –  Tom O'Connor Dec 10 '13 at 11:24

2 Answers 2

Configure nginx + php-fmp static pool and you will be able to have mostly stable ram usage.

If you use apache + php module, your max apache memory usage is MaxClients * average request ram usage. Count this and you will see, how much ram your setup are able to eat:) Alternatively you can limit MaxClients, but nginx + php-fmp works better.

At least make sure, that (max php proceses * php memory_limit) + max mysql memory usage from tuning-primer.sh is not much more then your vm RAM.

share|improve this answer
    
Thanks for the answer. But why it kills mysqld instead apache2? –  Igor Ostancov Dec 10 '13 at 12:27
    
I don't think Apache is the root cause here.. I think it's MySQL hogging all the RAM because the config file has been configured for a server with >2GB of RAM –  Tom O'Connor Dec 10 '13 at 12:35
    
It kills the one, that uses most ram per process. But most likely, that sum of ram used by all apache processes is much higher, then mysql. –  Kazimieras Aliulis Dec 10 '13 at 12:55
    
It is in your provided log:) Mysql rss 19582 pages of memory and sum of all apache rss memory pages are 166280. Even if big part of it overlaps because of shared libraries, it anyway eats much more ram then mysql. –  Kazimieras Aliulis Dec 10 '13 at 13:28
    
When killed mysql was taking: anon-rss:78328kB... So clearly not all ram in server. –  Kazimieras Aliulis Dec 10 '13 at 13:32

Based on your my.cnf, and the MySQL Calculator Tool at http://www.omh.cc/mycnf/

Your current configuration, with 70 connections will require 1.3GB of RAM.

As the Tappet Brothers say (or used to, before they retired), "That'll be your problem".

You need to either reduce the size of the connection pool (I'd start here.. ) or the size of your memory buffers..

I'd ask whether these are default values, which are probably unsuitable for a server with seriously low RAM (like a DO droplet), or if they've been recommended by another source.

One option would be to move your database server to a separate droplet, to give a better overall resource allocation.

share|improve this answer
    
Y U DOWNVOTE AND NOT LEAVE A COMMENT? –  Tom O'Connor Dec 10 '13 at 13:53
    
I've never got 70 connections at time. Max was 5. And Server down when it was not more than that value. Based on calc it requires 121mb for 5 connections with this config. –  Igor Ostancov Dec 10 '13 at 15:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.