4

I'm using cgroup to partition my processes and I'm getting Out Of Memory messages in my kernel logs.

However, I can't find which partition causes them. I've checked the memory controller cgroup but there are no obvious ways to use it.

The problem is, by the time I see 'task killed' message in system logs the task is dead, its /proc entry is gone and cgroup's tasks file doesn't have the pid of the killed task.

2
  • I thought you wanted to know what caused the OOM. Your problem seems to be about figuring out how the system responded to the OOM. These are two different things. Apr 21, 2012 at 3:08
  • I need to be able to recover from OOM (which might be caused by a variety of reasons).
    – Cyberax
    Apr 23, 2012 at 19:08

1 Answer 1

8

Answering my own question. I've used SystemTap to hook into the OOM killer:


#!/usr/bin/env stap
%{
#include <linux/cgroup.h>
%}

function find_mem_cgroup:string(task:long) %{
    struct cgroup *cgrp;
    struct task_struct *tsk = (struct task_struct *)((long)THIS->task);

    /* Initialize with an empty value */ 
    strcpy(THIS->__retvalue, "NULL");

    cgroup_lock();
    cgrp = task_cgroup(tsk, mem_cgroup_subsys_id);
    if (cgrp)
        cgroup_path(cgrp, THIS->__retvalue, MAXSTRINGLEN);
    cgroup_unlock();
%}

probe kernel.function("oom_kill_task") {
    cgroup = find_mem_cgroup($p)
    exename = kernel_string($p->comm)
    printf("pid\t%d\tmem-cgroup\t%s\texe-name\t%s\n", $p->pid, cgroup, exename)
}

Works like this:

cyberax@cybnb:~/work/shell$ sudo stap -g oom.stap
pid    3966    mem-cgroup    /task1/1/    exe-name    oom_generator.p

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .