Questions tagged [oom-killer]

oom-killer Out-Of-Memory killer in the Linux kernel

Filter by
Sorted by
Tagged with
0
votes
1answer
28 views

Slab SReclaimable memory can not be recycled?

CentOS Linux release 7.2.1511 (Core) Linux version 3.10.0-514.26.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP TueJul 4 15:04:05 UTC ...
0
votes
1answer
76 views

Kill the most memory-hungry process when memory gets low [closed]

Sometimes my Linux desktop, laptop, or server run out of RAM because the experimental software I work with occasionally tries to use more RAM than I have available. When this happens, the system and ...
0
votes
2answers
157 views

Kubernetes DaemonSet Pod exceeds memory limit but is not killed

We use kubernetes (specifically: openshift) to run our infrastructure in our team. One of the daemonsets, fluentd, is currently causing a lot of trouble, frequently taking full nodes down with huge ...
2
votes
0answers
48 views

What abbreviations in OOM Killer memory statistics report means?

When OOM Killer or kernel reports memory state it uses the next abbreviations Node 0 DMA: 26*4kB (M) 53*8kB (UM) 33*16kB (ME) 23*32kB (UME) 6*64kB (ME) 7*128kB (UME) 1*256kB (M) 2*512kB (ME) 0*...
1
vote
0answers
182 views

Linux out of memory killer kills process when there is more than enough memory available

We are running a 64-Bit Ubuntu and our 32GB of physical memory is split into 3 zones (DMA: 16MB, DMA32: 4GB and Normal: 30GB). According to the dmesg logs printed below our system ran out of free ...
1
vote
1answer
2k views

Why total-vm is is different than show in table?

I am running into oom_killer activity and cannot undesrand dmesg output [Fri Jan 11 02:11:48 2019] lowmem_reserve[]: 0 2811 64249 64249 [Fri Jan 11 02:11:48 2019] Node 0 DMA32 free:248704kB min:...
5
votes
3answers
2k views

Please help interpret OOM-Killer

We were testing our java application and it was killed by the OOM Killer, logs below. A couple of points: this is a java web application. It runs with a 190G heap. The server is a 240G total memory ...
0
votes
1answer
93 views

What are the specific column headers for the “Mem-Info” dump after “oom-killer”

I'm running a Debian 8 Jessie development server. 16GB mem. I've looked at the other posts about oom-killer, but none are really pertinent to what I'm asking here I'm trying to track down what ...
0
votes
0answers
205 views

Strange numbers in Oom killer log

First, the log: [Wed aug 22 01:11:31 2018] java invoked oom-killer: gfp_mask=0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null), order=0, oom_score_adj=0 [Wed aug 22 01:11:31 2018] java ...
2
votes
3answers
403 views

Stop a process (script) from invoking oom killer

I had a very unfortunate situation, where a bash script that contained a subtle error gone wild and took all available memory and then started killing other tasks (production apps!) to get even more. ...
2
votes
2answers
940 views

OOM killer shut down MySQL although ram usage seems low

Setup EC2 t2.large instance with Ubuntu 16.04 (2 Cores, 8GB RAM) Apache/2.4.18 (Ubuntu) 5.7.20-0ubuntu0.16.04.1 (Ubuntu) Currently this instance hosts multiple Drupal 6 & 7 sites, around 150 ...
5
votes
1answer
4k views

Is it recommended to set OOMScoreAdjust=1000 in systemd for a replaceable service?

systemd has the OOMScoreAdjust option, which allows to adjust the oom-killer score of the started process. To quote from the systemd documentation: OOMScoreAdjust= Sets the adjustment level ...
1
vote
0answers
441 views

Processes is getting killed randomly on REdHat

We have around java virtual machines with limited memory, and MongoDB. Deployed processes are randomly getting killed/terminated abruptly in time. No matter java process, nodejs or mongodb process. We ...
0
votes
1answer
251 views

Number of httpd processes and prefork.c

Hi I got OOM killer problems which caused: [mpm_prefork:error] [pid 18103] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process I had to restart the server to get it working. In /...
0
votes
1answer
624 views

VmRSS used to only about 25%, yet oom-killer strikes

I have a dedicated MySQL server equipped with 128 GB RAM. MySQL recently gets killed by the oom-killer, although MySQL is configured to use 95 GB in the worst case. In my research I came across this: ...
0
votes
1answer
148 views

Kernel killing processes with memory available

We have a CentOS5.8 server with 64Gb RAM running normally around 700Mb RAM usage. When I try to do a tar -czf into about a 22Gb archive, the pagecache climbs up to about 22Gb, then the kernel starts ...
7
votes
2answers
9k views

How does Docker deal with OOM killer and memory limits?

I have a docker container which starts a simple java (jgroups-based) application through a bash script. The java process is limited through Xmx by 128m, the container is allowed to use 256m (swap is ...
2
votes
1answer
679 views

Adjust oom_score at process launch

I've got a few servers that have begun oom-killing their backup processes and, while I understand that encountering the oom condition is quite bad in itself, I need this process to not die so that ...
2
votes
1answer
3k views

Oracle invoked oom-killer even when RAM and SWAP is plenty free?

Can you please tell me what cause this Oracle process killed? Seems like plenty of RAM free, and plenty of SWAP free. There followed few other oracle processes killed. The VM has the 16G of vMem and 8 ...
-2
votes
1answer
105 views

Is there a way for Apache to control how much memory it reserves?

Apache seems to have no control over how much memory it reserves. In MPM prefork you control the maximum number of processes you want to allow, but whatever number you use there, there is always a ...
1
vote
1answer
474 views

OOM score adjustements

On Ubuntu 12.04/Java 1.7 I have been reading/searching about OOM killer and generally understood what "proc" man page says especially oom_adj part. It all started because my important "MQ listeners" ...
67
votes
3answers
6k views

Rsync triggered Linux OOM killer on a single 50 GB file

I have a single 50 GB file on server_A, and I'm copying it to server_B. I run server_A$ rsync --partial --progress --inplace --append-verify 50GB_file root@server_B:50GB_file Server_B has 32 GB ...
0
votes
0answers
411 views

OOM Killer not triggering?

I'm working in an environment which has just been upgraded to Amazon Linux 2015.03 from Debian 6, quite a big jump. On the new platform a server instance has locked up a few times due to running out ...
1
vote
1answer
12k views

Understanding OOM killer logs

I run some processes inside docker container and I use memory limitation for this container. Sometimes some processes inside docker container got killed by OOM killer. I see that in syslog file: ...
8
votes
2answers
2k views

View full program command line arguments in OOM killer logs

Is it possible to view full program command line arguments in OOM killler logs ? What I see now in /var/log/syslog is Memory cgroup out of memory: Kill process 29187 (beam.smp) score 998 or sacrifice ...
5
votes
3answers
7k views

Mysql crashing, oom-killer, out of memory, tuning issues?

I just moved all of my websites to a new server with 4GB RAM. Almost immediately, mysql started crashing, and at one point, didn't reboot which caused a major outage (since I didn't notice until ...
7
votes
1answer
3k views

linux OOM-kill why?

. Greetings, fellow admins. I post in the hope of shedding light on the OOM-kills that haunt one of my company's machines. I cannot decide if they are legit OOMs or not. It's a Centos 6.x with kernel ...
17
votes
4answers
12k views

Linux oom situation (32 bit kernel)

I have continous oom&panic situation unresolved. I am not sure system fills up all the ram (36GB). Why this system triggered this oom situation? I suspect it as related to lowmem zone in 32 bit ...
1
vote
1answer
113 views

On SmartOS how to tell if process was killed for OOM

A daemon process of mine gets killed almost immediately, as soon as it starts receiving requests. The application log has absolutely no error output, even when I try to run it in foreground. I have ...
7
votes
1answer
8k views

MongoDB getting OOM killed

we are running a mongodb replicaset on three machines. All three machines have around 16GB but only 255MB Swap. Swappiness is left on it's default value 60. The machines are running CentOS 6.4. The ...
4
votes
1answer
2k views

Troubleshooting oom-killer using atop: Is it fixed or not?

I have a Debian system with 4GB of RAM, 2GB of swap and 2 CPUs. I've been experiencing a number of system failures involving oom-killer. The system will be working fine one minute, and the next will ...
16
votes
1answer
14k views

Scaling Logstash (with redis/elasticsearch)

Over a cluster of over 12 centos 5.8 servers, I deployed logstash using the native logstash shipper, which sends /var/log/*/*.log back to a central logstash server. We tried using rsyslogd as the ...