Questions tagged [oom]

Linux Out-of-memory Killer

Filter by
Sorted by
Tagged with
-1 votes
1 answer
25 views

AWS EC2 instance running out of memory running NodeJs application, cannot identify root cause because of log suppression

As stated in the title we have a NodeJS application running on an EC2 instance on AWS which sporadically runs out of memory and kills the process. Business wise it is not really a hindrance since ...
treskilion's user avatar
1 vote
0 answers
185 views

OOM killer beeing invoked by snapd systemd timer, CPU usage locks at 100% (snapd, mount, certbot)

Im having trouble debugging why my nodeJS webapp gets killed by the OOM killer in Debian 11. Im working on a small VPS (VMware ESX 7.0.3 build-21313628) by IONOS with single core 1GHz cpu, 512MB RAM, ...
nr1chiefrocka's user avatar
0 votes
0 answers
48 views

How to find memory consuming request in mongodb

There is a small(4 replicas) mongodb cluster(replicaset). Every node is 24GB ram with 8GB wiredtiger cache size. Cluster is used by many (about 100) applications which do few hundred requests per ...
undefine's user avatar
  • 1,046
0 votes
0 answers
235 views

Why does OOM killer works although memory is enough?

On one of the worker nodes of the Kubernetes cluster, a situation occurs periodically (several times a day) when the OOM killer is triggered, killing the "manager" process I assume it's ...
VladF's user avatar
  • 11
2 votes
2 answers
553 views

How to run DNF without going OOM on GCP e2-micro

I'm trying to run Fedora Cloud Base 36 on an e2-micro in the US central region of Google Cloud Platform (GCP). I haven't upgraded packages in a couple months so I tried to do that today. When I run: $ ...
jrefior's user avatar
  • 123
2 votes
0 answers
2k views

Explain of OOM killer logs

I have a question about the OOM killer logs. We are expecting a lot of OOM kills. The ecosystem My ecosystem looks like below: I have a server with 4 cores and 8 GB of RAM. I am running there the ...
Daniel Hornik's user avatar
0 votes
1 answer
659 views

Why MongoDB occurs out of memory (OOM) every few days

My Server Info AWS Ubuntu 18.04 MongoDB 4.2.17 4 Cores 16 GRAM This server only running mongodb, and allocated 10G memory to mongodb wiredTiger: engineConfig: cacheSizeGB: 10 But this ...
ikool's user avatar
  • 1
0 votes
0 answers
332 views

systemd out of memory log entry systemd-user:session

I have problem finding out reason of 'Out of memory' systemd log entry. Log for particular PID looks as follows - line 2: Sep 05 09:24:32 myServer systemd[24454]: pam_unix(systemd-user:session): ...
atapidriver's user avatar
0 votes
0 answers
92 views

Omm killer killed my python3 process when there was free memory

I am working on webserver on a linux embedded board without swap memory. Sometimes my webserver crashed when there was avaliable memory. I set those configurations to prevent this but it didn't work. ...
Selçuk Gök's user avatar
2 votes
1 answer
1k views

How to inspect why my service process be killed by Linux?

I have a vital service running a Ubuntu(20.04.1) server. Recently it is always killed by OS. At first I guess that probably is resulted by the OOM(out of memory) operation of OS, so I modified the ...
Leon's user avatar
  • 169
0 votes
0 answers
271 views

MariaDB memory much higher than innodb_buffer_pool_size

I have a backend server with 1G RAM for my HTTP server and my MariaDB. I noticed the database keeps getting killed by OOM once or twice a day. Most of the time the OOM is triggered by the HTTP server, ...
ydrezende's user avatar
2 votes
1 answer
328 views

oom_reaper: MongoDb out of memory

We run a small mongodb replica set on three bare metal servers (no virtualization, no docker/kubernetes) with Debian 11 and mongodb 5.0.6: machineA: 128GB RAM, 1TB disk, PRIMARY machineB: 128GB RAM, ...
mr.simonski's user avatar
0 votes
1 answer
196 views

Out of Memory Mismatch compared to what system rerports

Oct 25 07:41:32 KVM-BOX kernel: memory: usage 255216608kB, limit 255216640kB, failcnt 28058 [root@KVM-BOX ~]# free -hm total used free shared buff/cache available ...
Anonymous_Squirrel69's user avatar
3 votes
0 answers
504 views

Running out of memory with plenty of memory and swap

I have a Debian-based system that is running out of memory but there appears to be plenty of free memory. The box will run for about 6-12 days and then it will begin killing anything allocating memory ...
Dom G's user avatar
  • 31
0 votes
1 answer
574 views

mariadb oom-killer on CentOS8 in EC2 t2.micro instance

I seem to be having memory problems in my t2.micro instance (1GB) running nginx, mariadb, php and WordPress. I can see that mariadb.service is being killed regularly (I have used grep -e kill /var/log/...
dtw's user avatar
  • 103
0 votes
0 answers
872 views

RHEL 8. linux OOM killer invoked to kill postgres process

Database is going to recovery mode whenever the OOM killer is invoked on the postgres process. /var/log/messages output with respect to the issue oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),...
Sandesh Bailur's user avatar
0 votes
1 answer
705 views

Daily Apt Upgrade causing oom-killer to kill my java server

My server has died several times in the middle of the night because of this. How much memory is apt-upgrade asking for? Is it being reasonable? I'm trying to decipher the logs here. Any help or ...
satnam's user avatar
  • 51
0 votes
2 answers
381 views

Why cannot use buff/cache?

It seems mysqld allocates too much memory as buff/cache: free -m total used free shared buff/cache available Mem: 990 448 96 36 ...
William's user avatar
  • 99
0 votes
1 answer
656 views

When the kernel says "Out of memory: Killed proces..." does it mean real memory or virtual memory?

Looking at journal -xe for the reason(s) a process is killed, the message comes up in red "Out of memory: Killed process..." but it doesn't say whether the limiting resource is virtual ...
James Bowery's user avatar
6 votes
1 answer
724 views

fio 3.23 core dumps when bench-marking many small files

I have been asked to come up fio benchmark results for this test dataset: 1048576x1MiB. So, overall size is 1TiB. The set contains 2^20 1MiB files. The server runs CentOS Linux release 7.8.2003 (Core)...
foss4me's user avatar
  • 63
0 votes
1 answer
759 views

How do I track down a mysql memory issue on a server running a single WordPress site?

I'm having a weird issue. A few times a week, a website I'm running on Amazon Lightsail comes to a crashing halt and becomes IO bound (I can't usually use ssh when this happens) and the server runs ...
StevieD's user avatar
  • 534
3 votes
4 answers
6k views

mysqld crashing almost every hour - mysqld invoked oom-killer

I am running wordpress on a centOS digital ocean droplet my mysql keeps crashing almost every hour -- with an out of memory error manually restarting mysqld brings it back up via sudo service mysqld ...
Alicia R's user avatar
2 votes
2 answers
2k views

NFS server with out of memory

I have multiple machines sharing home directory via NFS share used by 6-10 users. All machines are used to run computational experiments including the one with NFS server. Although it is very rare but ...
rmah's user avatar
  • 29
0 votes
1 answer
2k views

Kubernetes Pod OOMKilled Issue

The scenario is we run some web sites based on an nginx image in kubernetes cluster. When we had our cluster setup with nodes of 2cores and 4GB RAM each. The pods had the following configurations, cpu:...
Shahnewaz Ul Islam Chowdhury's user avatar
0 votes
1 answer
925 views

Kill the most memory-hungry process when memory gets low [closed]

Sometimes my Linux desktop, laptop, or server run out of RAM because the experimental software I work with occasionally tries to use more RAM than I have available. When this happens, the system and ...
Vortico's user avatar
  • 271
0 votes
0 answers
1k views

Out of memory in Linux server killing my tomcat server

From last night I am getting below unexpected behaviour. Out of memory: Kill process 1740 (java) score 128 or sacrifice child Apr 8 19:14:20 ubuntu-s-4vcpu-8gb-sgp1-01 kernel: [17644.393549] Out of ...
Md. Sajedul Karim's user avatar
5 votes
1 answer
4k views

apache httpd process be killed by kernel oom

some apache httpd process's memory and cpu usage too high, be killed by kernel, but I don't know why httpd use such many memory. I need some method to analyse or some method to reproduct. some ...
user545235's user avatar
0 votes
0 answers
494 views

Why is systemd-resolve killing my java server?

I've been running into this problem where every few days one of my java servers is just dying. To figure out why, I ran cat /var/log/syslog which produces: Sep 21 06:36:24 ip-172-31-42-5 systemd-...
satnam's user avatar
  • 51
0 votes
1 answer
419 views

OOM despite the large amount of free SWAP

During some ML training htop shows usage of all RAM (16GB) and only 2G (out of 16GB) of SWAP when OOM happens. dmesg shows: [pon sie 12 11:53:44 2019] Purging GPU memory, 0 bytes freed, 131072 ...
3voC's user avatar
  • 103
0 votes
1 answer
1k views

What Java -Xmx setting can I reasonably expect to work on an AWS EC2 Ubuntu server?

What is your experience? Can you confirm my experimental findings? Can I generally use total RAM - 600 MB, or 0.4 * total RAM? Or is it always trial and error, and hoping that it is low enough? ...
Reto Höhener's user avatar
2 votes
0 answers
79 views

What abbreviations in OOM Killer memory statistics report means?

When OOM Killer or kernel reports memory state it uses the next abbreviations Node 0 DMA: 26*4kB (M) 53*8kB (UM) 33*16kB (ME) 23*32kB (UME) 6*64kB (ME) 7*128kB (UME) 1*256kB (M) 2*512kB (ME) 0*...
Raf's user avatar
  • 173
1 vote
0 answers
558 views

Linux out of memory killer kills process when there is more than enough memory available

We are running a 64-Bit Ubuntu and our 32GB of physical memory is split into 3 zones (DMA: 16MB, DMA32: 4GB and Normal: 30GB). According to the dmesg logs printed below our system ran out of free ...
Christof Länzlinger's user avatar
1 vote
3 answers
2k views

OOM kills a process while there is still a lot of available memory

I am aware that there are many threads like this one, but none of them actually give a methodology on how to read an OOM message. I have a db that is being monitored through NRPE for available ...
user3081519's user avatar
1 vote
1 answer
2k views

Java application gets killed in kubernetes while the resource limits and heap size are specified

Background A spring boot Java application is deployed in a kubernetes cluster and gets killed several times per day. I'm using openjdk:8u181-jre for my Java apps. Kubernetes version: v1.11.5 Node ...
leoleozhu's user avatar
  • 113
6 votes
2 answers
28k views

Enough memory but OOM killer invoked?

It has been a long time since my old friend the OOM killer came to visit me. Now it is coming by once a week and I cannot figure out why it’s triggered. It doesn’t seem that I’m missing memory. What ...
MikeSkril's user avatar
4 votes
2 answers
354 views

Out of memory at 18% usage : where the ram goes?

This is related to : Out of memory at 72% usage It looks to be the same problem but the question is slightly different : Where my memory goes ? I have 18% memory usage and my OOM Killer is killing ...
Ifnot's user avatar
  • 619
3 votes
2 answers
3k views

Linux OOM-killer acting despite plenty available memory

Once a week or so the OOM-killer shoots down a postgres process in my server, despite that 'free' states it has plenty of available memory. I have read several threads here and there, but cant see ...
Alexander Kolodziej's user avatar
1 vote
0 answers
697 views

OOM killer kills process though page cache is never evicted

Our servers in production sometimes under load runs into the following issue (Linux 4.10): Application handling the load (webserver) is running at high load. A new job starts (e.g. cron). This ...
Confused's user avatar
1 vote
0 answers
4k views

"bash: fork: Cannot allocate memory" for any command - how to prevent this at OS level?

I was testing how an Ubuntu 16.04 machine handles low memory conditions and had several suspended jobs that that used up almost all the RAM on the server. I was still able to run normal bash commands ...
EM0's user avatar
  • 370
3 votes
1 answer
6k views

Linux OOM: Trying to understand memory statistics / mismatch with expected values

I am trying to understand why the OOM killer has reaped a process on a KVM virtual machine running with 1 gigabyte of memory. While I do already know that running Linux machines without swap is not ...
ppmathis's user avatar
1 vote
0 answers
432 views

Is there any legitimate reason to mlock all of a process's mmaped files?

I'm administering some Linux servers that are running an httpd based application developed by one of our apps teams. This is in EC2 and each host has 32GB of RAM. I've noticed that there is about once ...
Michael Martinez's user avatar
3 votes
2 answers
9k views

Long page allocation stalls on Linux – why does this happen?

I have a problem (which I can reliably reproduce) on a bunch of Linux hosts, where the system becomes completely unresponsive after a process aggressively consumes memory. I see things like this in ...
obeattie's user avatar
  • 304
3 votes
2 answers
1k views

GKE's Container-Optimized OS Out-Of-Memory freezes

I have issue with Container-Optimized OS on GKE. If I run this simple command https://pastebin.com/raw/0WPAnAzn to consume all the RAM, at some point host freezes and doesn't respond to anything. ...
nailgun's user avatar
  • 59
0 votes
1 answer
379 views

Number of httpd processes and prefork.c

Hi I got OOM killer problems which caused: [mpm_prefork:error] [pid 18103] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process I had to restart the server to get it working. In /...
Luke Wenke's user avatar
0 votes
1 answer
1k views

VmRSS used to only about 25%, yet oom-killer strikes

I have a dedicated MySQL server equipped with 128 GB RAM. MySQL recently gets killed by the oom-killer, although MySQL is configured to use 95 GB in the worst case. In my research I came across this: ...
tombom's user avatar
  • 105
1 vote
4 answers
15k views

Ansible throws an “ ERROR! A worker was found in a dead state ” error

When I run a playbook that simply copies a directory from one place to another, ansible throws ERROR! A worker was found in a dead state Error. After some googling, it looks like this is caused by ...
Sudh33ra's user avatar
  • 179
2 votes
1 answer
9k views

OOM killer invoked despite tons of memory available? [duplicate]

I have a server where the OOM killer has been invoked once or twice almost each night since two weeks ago. The server should have more than enough memory it could free (cache/buffers, inactive_file ...
Moritz Bunkus's user avatar
10 votes
1 answer
8k views

Linux process killed even though enough memory available

I am investigating why two of our processes were killed by the Linux OOM killer - even though there seems to have been enough RAM and plenty of SWAP available at both times. When I interpret it as by ...
Patrick's user avatar
  • 323
3 votes
2 answers
4k views

Slab reclaimable memory is not freed when needed

Correct me if I am wrong, but to my understanding slab reclaimable holds cached kernel objects which can be freed if needed. So if application needs to allocate more space, even if the 'free' memory ...
Horkyze's user avatar
  • 151
0 votes
1 answer
519 views

OOM Killer on xenserver - 2832MB memory free - killing xapi

I am trying to understand why OOM killer starts killing processes when there is plenty of memory free on the server: Result of uname -a: Linux hostname 2.6.32.43-0.4.1.xs1.8.0.835.170778xen #1 SMP ...
W Khan's user avatar
  • 1