The OOM Killer

The Out-Of-Memory (OOM) killer is the Linux kernel’s last line of defense against memory exhaustion. When a page allocation cannot be satisfied even after every cheaper recourse has been tried — waking kswapd, direct reclaim, compaction, sixteen rounds of retries — and the kernel has no graceful way to fail the request, it would otherwise spin forever or deadlock. Rather than wedge the whole machine, the kernel deliberately picks one process, sends it SIGKILL, and reclaims its memory so the system can keep running. The decision and the killing live in mm/oom_kill.c; the entry point is out_of_memory(), the per-task scoring is oom_badness(), and a dedicated kernel thread called the oom_reaper tears the victim’s address space down asynchronously so that a corpse blocked on a lock cannot stall the recovery (all per the v6.12 source). This note covers the global (system-wide) OOM killer end to end: the exact allocator seam it hangs off, the gauntlet of bail-outs before anything dies, the badness arithmetic, the kill sequence, the reaper, the tuning knobs, how to decode a real dmesg OOM report field by field, and why the modern consensus is that the kernel killer arrives far too late and should be front-run by a PSI-driven userspace daemon.

Version pin. Everything below is read from Linux 6.12, a maintained long-term-support (LTS) branch; mainline is on the 7.x series as of 2026-09. Where 6.18 LTS or 7.1 differ, the difference is called out and dated. Behavioural claims come from the code, not from in-tree comments or Documentation/ prose — one place where those two disagree is flagged explicitly below.

This note is about when and how the kill happens. Two siblings own adjacent territory and are cross-linked rather than duplicated: OOM Score and Victim Selection goes deeper on the selection walk and its history, and cgroup OOM and memory.oom.group owns the per-cgroup OOM path. The reclaim machinery whose exhaustion precedes every OOM kill is Memory Reclaim Overview, with Direct Reclaim and kswapd and Background Reclaim for the two reclaim entry points.

Mental Model — A Function the Allocator Calls, Not a Daemon That Watches

The single most common misconception about the OOM killer is that it is a monitor. It is not. There is no thread polling free memory and deciding when things look bad. The OOM killer is a function the page allocator calls when it gives up, on the stack of whichever unlucky task happened to be allocating at that moment. That is why the first line of every OOM report names a process that is usually innocent: it is the invoker, not the culprit.

Every physical-page allocation enters alloc_pages(). The fast path takes a page from a free list if one is available above the zone’s low watermark. If that fails, the request falls into __alloc_pages_slowpath(), which wakes kswapd, performs direct reclaim in the caller’s own context, tries compaction for high-order requests, and loops. Only when should_reclaim_retry() decides that further retries are pointless does the allocator call __alloc_pages_may_oom(), and only if that function’s gauntlet of exemptions is cleared does it call out_of_memory().

flowchart TB
  ALLOC["alloc_pages / kmalloc<br/>fast path: page above low watermark?"]
  SLOW["__alloc_pages_slowpath<br/>wake kswapd, direct reclaim,<br/>compaction, retry"]
  RETRY{"should_reclaim_retry<br/>free + reclaimable could<br/>still clear the min watermark?<br/>and under 16 no-progress loops?"}
  MAYOOM["__alloc_pages_may_oom<br/>trylock oom_lock, re-check freelist,<br/>run the exemption gauntlet"]
  OOM["out_of_memory oc"]
  SELECT["select_bad_process<br/>scan every task, oom_badness scoring"]
  KILL["oom_kill_process then __oom_kill_process<br/>SIGKILL victim plus every task sharing its mm"]
  REAP["oom_reaper kthread<br/>fires 2 s later, unmaps anon VMAs"]
  FREE["pages returned to the buddy allocator"]
  ALLOC -->|"fail"| SLOW
  SLOW --> RETRY
  RETRY -->|"yes, retry"| SLOW
  RETRY -->|"no, give up"| MAYOOM
  MAYOOM -->|"no exemption applies"| OOM
  MAYOOM -->|"exempt: costly order,<br/>RETRY_MAYFAIL, THISNODE,<br/>lowmem, coredump"| NOPAGE["return NULL<br/>caller sees allocation failure"]
  OOM --> SELECT --> KILL
  KILL -->|"queue_oom_reaper"| REAP
  REAP --> FREE
  FREE --> ALLOC

The global OOM path from allocation to reaped memory. What it shows: the killer sits at the very bottom of the allocator slow path, behind two independent filters — the reclaim-retry loop, then the exemption gauntlet — and the memory it recovers comes back through a separate asynchronous thread, not through the kill itself. The insight: killing is fast (it is one signal) but freeing is not, because the victim must be scheduled to run its own exit path; the oom_reaper exists precisely because that assumption failed in practice. Note also the right-hand branch: many failing allocations never reach out_of_memory() at all and simply return NULL, which is why “the box was out of memory” and “the OOM killer fired” are different events.

Three consequences fall out of this shape, and they explain most confusing OOM behaviour:

1. An allocation that cannot sleep can never trigger the OOM killer. GFP_ATOMIC and GFP_NOWAIT clear __GFP_DIRECT_RECLAIM, so they never enter the slow path’s reclaim loop and never reach __alloc_pages_may_oom(). They just fail. See GFP Flags and Allocation Contexts.
2. The invoking task is usually not the memory hog. Whoever happens to need a page when the pool runs dry pays the cost of running the killer. In the worked example later in this note, the invoker is pipewire-pulse — an audio daemon — while the victim is a 39 GB Python process.
3. A single global mutex serialises the whole machine. __alloc_pages_may_oom() takes oom_lock with mutex_trylock(). Only one OOM event happens at a time, system-wide.

The Allocator Seam — Seven Ways to Avoid Killing Anything

__alloc_pages_may_oom() in mm/page_alloc.c is the only place the global killer is invoked from. Before it will consider blaming anybody, it runs a gauntlet. Reading it in order is the fastest way to understand why the OOM killer sometimes does not fire on a machine that is obviously out of memory.

flowchart TB
  A["__alloc_pages_may_oom entry"]
  B{"mutex_trylock oom_lock<br/>succeeds?"}
  C["another CPU is already OOM-killing:<br/>set did_some_progress = 1,<br/>schedule_timeout_uninterruptible 1 tick,<br/>return NULL so the caller retries"]
  D{"page available at the<br/>HIGH watermark with<br/>direct reclaim masked off?"}
  E["a parallel kill or free already<br/>produced memory: return the page,<br/>nobody dies"]
  F{"current has PF_DUMPCORE?"}
  G{"order greater than<br/>PAGE_ALLOC_COSTLY_ORDER = 3?"}
  H{"gfp has __GFP_RETRY_MAYFAIL<br/>or __GFP_THISNODE?"}
  I{"highest_zoneidx below<br/>ZONE_NORMAL?"}
  J{"pm_suspended_storage?"}
  K["out_of_memory oc<br/>'Exhausted what can be done<br/>so it is blame time'"]
  X["goto out: unlock, return NULL"]
  A --> B
  B -->|"no"| C
  B -->|"yes"| D
  D -->|"yes"| E
  D -->|"no"| F
  F -->|"yes"| X
  F -->|"no"| G
  G -->|"yes"| X
  G -->|"no"| H
  H -->|"yes"| X
  H -->|"no"| I
  I -->|"yes"| X
  I -->|"no"| J
  J -->|"yes"| X
  J -->|"no"| K

The exemption gauntlet in __alloc_pages_may_oom(), in source order. What it shows: six independent conditions, any one of which turns “out of memory” into a quiet NULL return rather than a kill. The insight: the OOM killer is invoked only for allocations that are both small and unable to fail. If you are debugging a page allocation failure: order:4 splat with no kill, this diagram is the answer — order 4 is above PAGE_ALLOC_COSTLY_ORDER, and the killer would not have helped anyway, because scattering free pages does not create a contiguous eight-page run.

Each exemption has a reason, and the in-tree comments state them:

Exemption	Condition	Why the kernel declines to kill
Already OOM-ing	mutex_trylock(&oom_lock) fails	Somebody else “is making progress for us”. Launching a second concurrent kill would double the casualties for one shortage.
Race won	get_page_from_freelist() succeeds at ALLOC_WMARK_HIGH	A parallel kill or free already released memory. The re-check deliberately masks off __GFP_DIRECT_RECLAIM so it cannot recurse while holding oom_lock.
Core dumping	current->flags & PF_DUMPCORE	“Coredumps can quickly deplete all memory reserves.” The dump is already in progress; killing here corrupts it.
Costly order	order > PAGE_ALLOC_COSTLY_ORDER (order 3 = 8 pages)	“The OOM killer will not help higher order allocs.” Freeing scattered pages does not produce contiguous runs — that is compaction’s job.
Caller has a fallback	__GFP_RETRY_MAYFAIL or __GFP_THISNODE	“It is very likely that the caller has a more reasonable fallback than shooting a random task.” __GFP_THISNODE additionally cannot be helped: “the OOM killer may not free memory on a specific node.”
Lowmem-only request	ac->highest_zoneidx < ZONE_NORMAL	“The OOM killer does not needlessly kill tasks for lowmem.” Killing a task in ZONE_NORMAL does not refill ZONE_DMA.
Storage suspended	pm_suspended_storage()	Swap and filesystems are unavailable mid-suspend; the victim could not free anything through them.

One condition is conspicuously absent from that list and catches people out. GFP_NOFS allocations are not exempted here. The comment says so, and apologises:

/*
 * XXX: GFP_NOFS allocations should rather fail than rely on
 * other request to make a forward progress.
 * We are in an unfortunate situation where out_of_memory cannot
 * do much for this context but let's try it to at least get
 * access to memory reserved if the current task is killed (see
 * out_of_memory). Once filesystems are ready to handle allocation
 * failures more gracefully we should just bail out here.
 */

The bail-out for GFP_NOFS happens one level down, inside out_of_memory() itself — and it happens before any scoring, which is the subject of the next section.

The oom_control struct handed down carries the allocation’s whole context: zonelist, nodemask, gfp_mask, order, and .memcg = NULL marking this as a global rather than cgroup-scoped event. The order field doubles as a sentinel: order == -1 means “this OOM was requested by SysRq”, checked via is_sysrq_oom(), which exempts the event from panic_on_oom and from the “already a victim in flight” abort.

Inside out_of_memory() — Nine Gates Before Anybody Dies

out_of_memory() is the orchestrator, and it is best read as an ordered sequence of gates. Here is the body from mm/oom_kill.c at v6.12, lightly elided:

bool out_of_memory(struct oom_control *oc)
{
	unsigned long freed = 0;
 
	if (oom_killer_disabled)                                   /* gate 1 */
		return false;
 
	if (!is_memcg_oom(oc)) {                                   /* gate 2 */
		blocking_notifier_call_chain(&oom_notify_list, 0, &freed);
		if (freed > 0 && !is_sysrq_oom(oc))
			/* Got some memory back in the last second. */
			return true;
	}
 
	if (task_will_free_mem(current)) {                         /* gate 3 */
		mark_oom_victim(current);
		queue_oom_reaper(current);
		return true;
	}
 
	if (!(oc->gfp_mask & __GFP_FS) && !is_memcg_oom(oc))       /* gate 4 */
		return true;
 
	oc->constraint = constrained_alloc(oc);                    /* gate 5 */
	if (oc->constraint != CONSTRAINT_MEMORY_POLICY)
		oc->nodemask = NULL;
	check_panic_on_oom(oc);                                    /* gate 6 */
 
	if (!is_memcg_oom(oc) && sysctl_oom_kill_allocating_task && /* gate 7 */
	    current->mm && !oom_unkillable_task(current) &&
	    oom_cpuset_eligible(current, oc) &&
	    current->signal->oom_score_adj != OOM_SCORE_ADJ_MIN) {
		get_task_struct(current);
		oc->chosen = current;
		oom_kill_process(oc, "Out of memory (oom_kill_allocating_task)");
		return true;
	}
 
	select_bad_process(oc);                                    /* gate 8 */
	if (!oc->chosen) {
		dump_header(oc);
		pr_warn("Out of memory and no killable processes...\n");
		if (!is_sysrq_oom(oc) && !is_memcg_oom(oc))
			panic("System is deadlocked on memory\n");
	}
	if (oc->chosen && oc->chosen != (void *)-1UL)              /* gate 9 */
		oom_kill_process(oc, !is_memcg_oom(oc) ? "Out of memory" :
				 "Memory cgroup out of memory");
	return !!oc->chosen;
}

Walking the gates in order:

Gate 1 — oom_killer_disabled. A global flag, set only by oom_killer_disable(), which is called by the hibernation/suspend path. While it is set, every allocation that would have OOM-ed fails instead. oom_killer_disable() takes oom_lock (killably), sets the flag, and then waits on oom_victims_wait until the oom_victims counter drains to zero — i.e. it blocks until every in-flight victim has finished dying — with a timeout, and re-enables the killer if the timeout expires. The header comment is blunt about how special this is: “The function cannot be called when there are runnable user tasks because the userspace would see unexpected allocation failures as a result. Any new usage of this function should be consulted with MM people.”

Gate 2 — the OOM notifier chain. blocking_notifier_call_chain(&oom_notify_list, 0, &freed) gives registered subsystems one last chance to cough up pages, and each returns how many it freed in freed. If any did, out_of_memory() returns true and nobody dies. Registration is via the exported register_oom_notifier(). Balloon drivers (virtio-balloon and friends) are the archetypal users: a hypervisor guest under pressure can deflate its balloon and hand pages back to itself rather than kill a process. Note the callback runs from a blocking notifier chain, so it is allowed to sleep, and that a sysrq-triggered OOM ignores the result — a manual OOM request is meant to kill something.

Gate 3 — the current task is already dying. task_will_free_mem(current) checks whether the caller is on its way out and will free its memory unaided: no core dump in progress (sig->core_state), either SIGNAL_GROUP_EXIT is set or it is a single-threaded task with PF_EXITING, MMF_OOM_SKIP is not already set (i.e. the reaper has not already drained it), and — if the mm is shared — every other process sharing that mm is also dying. If all that holds, there is no point picking a fresh victim: mark the caller an OOM victim (which grants it reserve access so it can finish exiting) and queue the reaper to help.

Gate 4 — GFP_NOFS gets reserves, not a kill. if (!(oc->gfp_mask & __GFP_FS) && !is_memcg_oom(oc)) return true;. The comment reads: “The OOM killer does not compensate for IO-less reclaim. But mem_cgroup_oom() has to invoke the OOM killer even if it is a GFP_NOFS allocation.” This is a rule with real operational consequences: a filesystem allocation made inside a GFP_NOFS (or memalloc_nofs_save()) scope will never kill anything at global scope. It returns true, the allocator treats that as “progress was made”, and the request loops back around. The reasoning is that a NOFS allocation has deliberately hobbled reclaim, so its failure is not evidence that the machine is out of memory — it is evidence that this caller forbade the tools that would have found memory. Compare GFP Flags and Allocation Contexts for what __GFP_FS actually gates.

Gate 5 — constrained_alloc() classifies the shortage and sets the denominator. This is the step people most often miss, and it is why a process can be OOM-killed on a NUMA machine that shows plenty of free RAM. The function returns one of four constraints and, critically, sets oc->totalpages, the denominator against which every task’s footprint is judged:

Constraint	Trigger	oc->totalpages set to
CONSTRAINT_MEMCG	oc->memcg != NULL	mem_cgroup_get_max(oc->memcg) ?: 1 — the cgroup’s limit
CONSTRAINT_MEMORY_POLICY	oc->nodemask is a strict subset of node_states[N_MEMORY] (an mbind/set_mempolicy node set)	total_swap_pages + the sum of node_present_pages(nid) over the policy’s nodes
CONSTRAINT_CPUSET	some zone in the zonelist fails cpuset_zone_allowed()	total_swap_pages + the sum of node_present_pages(nid) over cpuset_current_mems_allowed
CONSTRAINT_NONE	none of the above, or !CONFIG_NUMA, or __GFP_THISNODE	totalram_pages() + total_swap_pages — the whole machine

The __GFP_THISNODE case is a deliberate surrender, and the comment says so: “Reach here only when __GFP_NOFAIL is used. So, we should avoid to kill current. We have to random task kill in this case. Hopefully, CONSTRAINT_THISNODE… but no way to handle it, now.” There is no per-node constraint type, so a single-node exhaustion is scored against the whole machine’s memory, which systematically under-scores everybody and makes the choice close to arbitrary.

The constraint string is printed verbatim in the OOM report (oom-kill:constraint=CONSTRAINT_NONE,...), so this table is directly actionable when reading a log.

Gate 6 — check_panic_on_oom(). Covered in the configuration section below. It runs before oom_kill_allocating_task, which is why vm.rst states that “if panic_on_oom is selected, it takes precedence over whatever value is used in oom_kill_allocating_task.”

Gate 7 — oom_kill_allocating_task. With this sysctl non-zero, the killer skips the tasklist scan entirely and kills current. Note the four extra conditions the code requires that the documentation does not mention: the caller must have an mm (not a kernel thread mid-kthread_use_mm()), must not be oom_unkillable_task(), must be cpuset-eligible for the failing allocation, and must not carry oom_score_adj == -1000. If any fails, the code falls through to the normal scan rather than doing nothing.

Gate 8 — select_bad_process(). The tasklist scan, detailed in the next section.

Gate 9 — the kill, or a panic. If the scan chose nobody, a global allocation has nowhere to go: the allocator would loop forever, so the kernel calls panic("System is deadlocked on memory\n"). A memcg OOM or a SysRq OOM merely logs and returns. This is the mechanism behind the most self-inflicted OOM outage there is — protecting so many processes with oom_score_adj = -1000 that no killable candidate remains, converting a survivable single-process kill into a machine-wide panic.

Scoring — What oom_badness() Actually Counts

The scoring function is deliberately, almost defiantly simple. Its kernel-doc says so: “The heuristic for determining which task to kill is made to be as simple and predictable as possible. The goal is to return the highest value for the task consuming the most memory to avoid subsequent oom failures.” Here it is in full at v6.12:

long oom_badness(struct task_struct *p, unsigned long totalpages)
{
	long points;
	long adj;
 
	if (oom_unkillable_task(p))                 /* PID 1 or a kernel thread */
		return LONG_MIN;
 
	p = find_lock_task_mm(p);                   /* find a thread that still has an mm */
	if (!p)
		return LONG_MIN;
 
	adj = (long)p->signal->oom_score_adj;
	if (adj == OOM_SCORE_ADJ_MIN ||             /* -1000: opted out entirely */
			test_bit(MMF_OOM_SKIP, &p->mm->flags) ||  /* already reaped */
			in_vfork(p)) {                      /* borrowing the parent's mm */
		task_unlock(p);
		return LONG_MIN;
	}
 
	points = get_mm_rss(p->mm) + get_mm_counter(p->mm, MM_SWAPENTS) +
		mm_pgtables_bytes(p->mm) / PAGE_SIZE;
	task_unlock(p);
 
	adj *= totalpages / 1000;                   /* normalise to oom_score_adj units */
	points += adj;
 
	return points;
}

Three exclusions and three addends. Walk them symbol by symbol.

The exclusions all return LONG_MIN, the smallest possible long, making the task permanently the least attractive candidate:

• oom_unkillable_task(p) is true for is_global_init(p) (PID 1 — killing init panics the box) and for p->flags & PF_KTHREAD (kernel threads have no user address space to reclaim).
• oom_score_adj == OOM_SCORE_ADJ_MIN (-1000, from include/uapi/linux/oom.h) is the supported userspace opt-out.
• MMF_OOM_SKIP marks an mm the reaper has already drained (or given up on). Excluding it is what stops the killer from re-picking the same corpse and going on a spree while the first victim’s pages are still draining.
• in_vfork(p) — a vfork() child is temporarily running on its parent’s mm; killing it would destroy the parent’s address space.

The addends are all in units of pages:

• get_mm_rss(mm) — the resident set, defined in include/linux/mm.h as MM_FILEPAGES + MM_ANONPAGES + MM_SHMEMPAGES. Note that file-backed and shared-memory resident pages count, not just anonymous ones — which is the mechanical reason a process that merely mmaps a huge file can out-score a genuine leaker.
• get_mm_counter(mm, MM_SWAPENTS) — pages this task has pushed out to swap. Counting them is what stops a process from hiding its footprint by swapping.
• mm_pgtables_bytes(mm) / PAGE_SIZE — the page tables themselves, converted from bytes to pages. On a process with a sparse 380 TB virtual address space this is not a rounding error: in the worked example below, one process carries 74.6 MB of page tables, about 19 thousand pages.

What is deliberately not counted: total_vm (virtual size — a process can reserve terabytes it never touches, and Brave does exactly that), kernel memory charged on the task’s behalf (sockets, dentries — those are memcg’s business, see The Memory Cgroup memcg), shared pages counted once per sharer rather than once in total (two processes sharing a 1 GB mapping each score the full gigabyte), and anything about rate — a process leaking 100 MB/s and a static process holding the same RSS score identically.

The adjustment is the interesting arithmetic. adj *= totalpages / 1000 converts the userspace oom_score_adj value from “tenths of a percent of allowed memory” into an absolute page count, then adds it. So oom_score_adj = +500 adds half of totalpages to the score — as Documentation/filesystems/proc.rst puts it, “roughly equivalent to allowing the remainder of tasks sharing the same system, cpuset, mempolicy, or memory controller resources to use at least 50% more memory.”

flowchart LR
  subgraph COUNT["counted, in pages"]
    R["get_mm_rss<br/>= MM_FILEPAGES<br/>+ MM_ANONPAGES<br/>+ MM_SHMEMPAGES"]
    S["MM_SWAPENTS<br/>pages pushed to swap"]
    P["mm_pgtables_bytes / PAGE_SIZE<br/>the page tables themselves"]
  end
  subgraph BIAS["userspace bias"]
    A["oom_score_adj<br/>times totalpages / 1000"]
  end
  subgraph IGNORED["deliberately ignored"]
    V["total_vm<br/>virtual reservations"]
    K["kernel memory charged<br/>on the task's behalf"]
    D["allocation RATE<br/>leak vs steady state"]
    SH["sharing: each sharer is<br/>charged the full mapping"]
  end
  R --> SUM["points"]
  S --> SUM
  P --> SUM
  A --> SUM
  SUM --> OUT["long badness<br/>LONG_MIN if unkillable,<br/>vfork, MMF_OOM_SKIP,<br/>or adj = -1000"]

What oom_badness() weighs, at v6.12. What it shows: the score is three page counters plus one linear bias, with four hard exclusions — that is the entire heuristic. The insight: because oom_score_adj is scaled by totalpages/1000, a single-digit-percent adjustment is worth gigabytes on a large machine, and routinely dominates the actual memory measurement. On the 128 GB box used in the worked example, +100 of adjustment is worth 3.5 million pages — about 13 GB — which is more resident memory than any process on the system except the runaway one.

Documentation and code disagree here — trust the code

Documentation/filesystems/proc.rst at v6.12 still describes the badness heuristic as assigning “a value to each candidate task ranging from 0 (never kill) to 1000 (always kill)”, and says -1000 means the task “will always report a badness score of 0”. That was true of the pre-4.x normalised heuristic. In v6.12 oom_badness() returns raw page counts, unbounded above, and returns LONG_MIN — not 0 — for oom_score_adj == -1000. The 0–1000 normalisation survives only in fs/proc/base.c::proc_oom_score(), which rescales for the /proc/<pid>/oom_score file. Verified by reading both files at the v6.12 tag, 2026-09-04.

oom_score_adj, oom_score, and the legacy oom_adj

Userspace tunes selection through three /proc files, only one of which you should use.

File	Direction	Range	Meaning at v6.12
/proc/<pid>/oom_score_adj	read/write	-1000 … +1000	The supported knob. Added to the badness score after scaling by totalpages/1000. -1000 = never kill.
/proc/<pid>/oom_score	read-only	0 … 2000	Derived view of the current badness: points = (1000 + badness * 1000 / totalpages) * 2 / 3, per proc_oom_score(). Special-cased to 0 when badness is LONG_MIN.
/proc/<pid>/oom_adj	read/write	-16 … +15, plus -17 = OOM_DISABLE	Deprecated pre-2.6.36 interface. Writes are rescaled onto oom_score_adj (oom_adj * 1000 / 17, with +15 clamped to +1000) and log pr_warn_once(".../oom_adj is deprecated, please use .../oom_score_adj instead").

Two behaviours of oom_score_adj writes are worth knowing because they surprise people, and both live in __set_oom_adj() in fs/proc/base.c:

• There is a floor, and it is sticky. Each signal_struct carries an oom_score_adj_min alongside the value. When a process holding CAP_SYS_RESOURCE writes oom_score_adj, the write also lowers oom_score_adj_min to that value. Thereafter an unprivileged writer attempting to go below the floor gets -EACCES. This is how a service manager can pin a daemon’s protection without the daemon being able to reduce it further.
• The write propagates across an mm. If the target’s mm has MMF_MULTIPROCESS set (it is shared with a process outside the thread group) and the target is not a vfork() child, the new value is written to every process sharing that mm, skipping kernel threads and PID 1. Since oom_badness() scores an mm, letting two sharers disagree about protection would be meaningless.

The design rationale for treating this knob as coarse comes straight from the maintainer. Arguing in 2018 against extending oom_score_adj-style tuning to cgroups, Michal Hocko wrote (LWN, “Teaching the OOM killer about control groups”):

oom_score_adj is basically unusable for any fine tuning on the process level for most setups except for very specialized ones. The only reasonable usage I’ve seen so far was to disable OOM killer for a process or make it a prime candidate.

That is the right way to use it in production: -1000 for a handful of things that must never die, +1000 for a designated sacrificial process, and nothing in between.

The selection walk

select_bad_process() is a plain scan. For a global OOM it takes rcu_read_lock() and iterates for_each_process(p), calling oom_evaluate_task() on each; for a memcg OOM it delegates to mem_cgroup_scan_tasks(). oom_evaluate_task() filters, scores, and keeps a running maximum in oc->chosen/oc->chosen_points, seeded at LONG_MIN. Two behaviours are not obvious from the name:

• oom_task_origin() short-circuits everything with points = LONG_MAX. A task can call set_current_oom_origin() to declare “if I trigger an OOM, kill me first”. swapoff(2) is the canonical user: it pulls the entire contents of a swap device back into RAM, so if that causes an OOM the honest answer is to abort the swapoff rather than shoot an innocent workload.
• An existing victim aborts the whole scan. If any candidate is already an OOM victim (tsk_is_oom_victim()) and its mm does not yet carry MMF_OOM_SKIP, oom_evaluate_task() returns non-zero, oc->chosen is set to the sentinel (void *)-1UL, and the scan stops. out_of_memory() then returns true without killing. The comment: “This task already has access to memory reserves and is being killed. Don’t allow any other task to have access to the reserves.” Wait for the first corpse before making a second.

The deeper history of the heuristic — why it looks like this, what earlier versions weighed, and the repeatedly-rejected proposals to make it cgroup-aware — is in OOM Score and Victim Selection.

The Kill Sequence

oom_kill_process() and __oom_kill_process() do the deed. The order of operations is deliberate in ways that are easy to misread.

sequenceDiagram
    autonumber
    participant AL as "allocating task<br/>(the invoker)"
    participant OOM as "out_of_memory / oom_kill_process"
    participant NOT as "OOM notifier chain"
    participant V as "victim task"
    participant SH as "tasks sharing victim mm"
    participant RT as "reaper timer (2 s)"
    participant RK as "oom_reaper kthread"
    participant BUD as "buddy allocator"

    AL->>OOM: __alloc_pages_may_oom, holding oom_lock
    OOM->>NOT: blocking_notifier_call_chain
    NOT-->>OOM: freed = 0, nothing recovered
    OOM->>OOM: constrained_alloc sets totalpages
    OOM->>OOM: select_bad_process scans tasklist
    Note over OOM: victim chosen, chosen_points recorded
    OOM->>OOM: task_will_free_mem victim? if yes, just mark and queue
    OOM->>OOM: dump_header, ratelimited: gfp_mask, Mem-Info, task table
    OOM->>OOM: mem_cgroup_get_oom_group victim
    OOM->>V: do_send_sig_info SIGKILL, PIDTYPE_TGID
    OOM->>V: mark_oom_victim: TIF_MEMDIE, pin oom_mm, __thaw_task
    Note over OOM,V: SIGKILL is sent BEFORE reserve access is granted,<br/>so the victim cannot spend reserves from userspace
    OOM->>OOM: pr_err "Killed process ... anon-rss ... oom_score_adj"
    OOM->>SH: SIGKILL every process sharing the mm (no reserve access)
    OOM->>RT: queue_oom_reaper, arm timer for jiffies + 2*HZ
    OOM-->>AL: return true, unlock oom_lock, allocator retries
    V--)V: scheduled, runs exit path, unmaps what it can
    RT->>RK: wake_oom_reaper, unless MMF_OOM_SKIP already set
    RK->>V: mmap_read_trylock, up to 10 attempts, 100 ms apart
    RK->>BUD: unmap_page_range over anon and private VMAs
    RK->>RK: set MMF_OOM_SKIP, log "oom_reaper: reaped process ..."
    BUD-->>AL: pages available, allocation finally succeeds

The full kill-to-recovery timeline. What it shows: the ordering constraints that the code comments spell out — notifier first, SIGKILL before reserve access, whole-mm kill before the reaper is armed, and a two-second gap before the reaper touches anything. The insight: the allocating task does not wait for memory to come back. out_of_memory() returns true, meaning “progress was made”, and the allocator loops. Recovery is a race between the victim’s own exit path and a timer; the reaper is the backstop for when the former loses.

Four details in __oom_kill_process() reward attention:

SIGKILL precedes mark_oom_victim(), on purpose. The comment: “We should send SIGKILL before granting access to memory reserves in order to prevent the OOM victim from depleting the memory reserves from the user space under its control.” mark_oom_victim() sets TIF_MEMDIE and pins tsk->signal->oom_mm, and __gfp_pfmemalloc_flags() in the allocator turns that into ALLOC_OOM, which halves the effective min watermark again (see Watermarks and the Allocation Fast Path). A victim that could still run userspace code with that privilege could drain the reserves the rest of the machine needs.

mark_oom_victim() thaws a frozen task. __thaw_task(tsk) wakes the victim if the cgroup freezer or a suspend cycle had it parked, “because OOM killer wouldn’t be able to free any memory and livelock” otherwise.

The whole mm dies, but only the chosen task gets reserves. After the primary kill, the code walks for_each_process(p) and SIGKILLs every process that process_shares_mm(p, mm) outside the victim’s thread group. Those get no reserve access — “to avoid depletion of all memory” — but the comment explains why they must die at all: “This prevents mm->mmap_lock livelock when an oom killed thread cannot exit because it requires the semaphore and it’s contended by another thread trying to allocate memory itself.” Killing all sharers turns their allocations into fatal-signal-pending allocations, which are allowed to fail out of the loop.

PID 1 sharing the victim’s mm cancels reaping. If is_global_init(p) is among the sharers, the code sets can_oom_reap = false, sets MMF_OOM_SKIP on the mm, and logs oom killer <pid> (<comm>) has mm pinned by <pid> (<comm>). Init cannot be killed, so the address space will never be fully released, and the reaper must not touch memory init might still read.

Group kill: memory.oom.group applies to global OOMs too. Before killing, oom_kill_process() calls mem_cgroup_get_oom_group(victim, oc->memcg) — and oc->memcg is NULL for a global OOM, which mem_cgroup_get_oom_group() substitutes with root_mem_cgroup. It then walks from the victim’s own cgroup upward to the OOM domain and returns the highest-level ancestor with oom_group set. If one is found, every task in that subtree is killed after the victim, excepting oom_score_adj == -1000 tasks and PID 1, and a MEMCG_OOM_GROUP_KILL event is raised on memory.events. In other words: setting memory.oom.group=1 on a container changes what a host-wide OOM kill does to that container, not just what a container-limit OOM does. The per-cgroup path is cgroup OOM and memory.oom.group.

The oom_reaper — Why Killing Is Not Freeing

Sending SIGKILL does not free a single page. The victim is an ordinary task: to release memory it must be scheduled, run do_exit() → exit_mm() → exit_mmap(), unmap its VMAs, and drop its page references. And an OOM victim is very frequently blocked — waiting on mmap_lock, or a filesystem lock, held by another task that is itself stuck in the page allocator waiting for memory. That is a textbook livelock: the killer fired, but the corpse cannot move, so no memory comes back, so the allocator OOMs again.

The oom_reaper closes that hole. It was introduced by Michal Hocko in commit aac4536355469 (“mm, oom: introduce oom reaper”, dated 2016-03-25), credited in the changelog to an idea from Mel Gorman at LSFMM 2015 and independently from Oleg Nesterov, with the motivating workloads constructed by Tetsuo Handa. It first shipped in Linux 4.6 — verified directly by existence-check: mm/oom_kill.c at the v4.5 tag contains zero occurrences of oom_reaper, and at v4.6 it contains 29.

The commit message states the core observation and the design constraint:

the OOM victim might take unbounded amount of time to exit because it might be blocked in the uninterruptible state waiting for an event (e.g. lock) which is blocked by another task looping in the page allocator.

A kernel thread has been chosen because we need a reliable way of invocation so workqueue context is not appropriate because all the workers might be busy (e.g. allocating memory). Kswapd which sounds like another good fit is not appropriate as well because it might get blocked on locks during reclaim as well.

The insight that makes it safe is that a task which has received SIGKILL will never run in user mode again, so it will never read its own anonymous pages again; those pages can be thrown away immediately without changing the outcome (LWN, Corbet, 2015-12-16).

stateDiagram-v2
    [*] --> Running: ordinary task
    Running --> Selected: select_bad_process picks it
    Selected --> Victim: SIGKILL sent,<br/>mark_oom_victim sets TIF_MEMDIE,<br/>oom_mm pinned, __thaw_task
    Victim --> Queued: queue_oom_reaper<br/>MMF_OOM_REAP_QUEUED set,<br/>timer armed for 2 seconds
    Queued --> ExitedFirst: victim ran exit_mmap first<br/>and set MMF_OOM_SKIP
    ExitedFirst --> [*]: wake_oom_reaper sees MMF_OOM_SKIP,<br/>drops the reference, does nothing
    Queued --> Listed: timer fires, pushed onto<br/>oom_reaper_list, kthread woken
    Listed --> Reaping: mmap_read_trylock succeeded
    Listed --> Retrying: trylock failed
    Retrying --> Listed: schedule_timeout_idle HZ/10,<br/>attempt under 10
    Retrying --> GaveUp: 10 attempts exhausted
    Reaping --> Reaped: all VMAs walked,<br/>anon and private pages unmapped
    Reaping --> Retrying: mmu_notifier nonblock start failed<br/>on some VMA, partial work
    Reaped --> [*]: MMF_OOM_SKIP set,<br/>"oom_reaper: reaped process ..."
    GaveUp --> [*]: MMF_OOM_SKIP set anyway,<br/>"oom_reaper: unable to reap pid:...",<br/>sched_show_task + debug_show_all_locks

The lifecycle of an OOM victim’s address space, from selection to reaped. What it shows: every terminal state sets MMF_OOM_SKIP, including the failure state. The insight: MMF_OOM_SKIP is not “reaping succeeded” — it means “the OOM killer must stop looking at this mm.” Setting it even on failure is what guarantees forward progress: a victim the reaper cannot drain becomes invisible to oom_badness(), so the next OOM event picks somebody else instead of deadlocking on the same unresponsive corpse.

The mechanism, function by function:

queue_oom_reaper(tsk) does not enqueue. It sets MMF_OOM_REAP_QUEUED on the victim’s mm with test_and_set_bit() (so a victim is queued at most once), takes a task reference, and arms a timer for jiffies + OOM_REAPER_DELAY, where #define OOM_REAPER_DELAY (2*HZ) — two seconds.

That delay is not an original design choice; it is a bug fix, and its story is worth knowing. It arrived in Linux 5.18 (existence-checked: absent from mm/oom_kill.c at v5.17, present at v5.18) via commit e4a38402c36e, “oom_kill.c: futex: delay the OOM reaper to allow time for proper futex cleanup”, from Joel Savitz and Nico Pache at Red Hat, based on a patch by Michal Hocko. glibc allocates the pthread struct — which holds the robust futex list head — in PRIVATE|ANONYMOUS memory, exactly the memory the reaper targets. The kernel does not keep its own copy of the robust list; on process death it reads the list from userspace memory to wake anyone holding a robust mutex. The race:

    CPU1                               CPU2
    --------------------------------------------------------------------
    page_fault
    do_exit "signal"
    wake_oom_reaper
                                        oom_reaper
                                        oom_reap_task_mm (invalidates mm)
    exit_mm
    exit_mm_release
    futex_exit_release
    futex_cleanup
    exit_robust_list
    get_user (EFAULT- can't access memory)

If that get_user() faults, “the kernel will be unable to recover the waiters on the robust_list, leaving userspace mutexes hung indefinitely.” The two-second grace period gives the exit path time to finish futex cleanup before the reaper starts destroying the memory it needs. The in-tree comment generalises the trade-off: “The timers timeout is arbitrary… the longer it is, the longer the worst case scenario for the OOM can take. If it is too small, the oom_reaper can get in the way and release resources needed by the process exit path.”

wake_oom_reaper() (the timer callback) first re-checks MMF_OOM_SKIP: if the victim already finished exiting on its own — the common case on a healthy system — the reaper does nothing and drops its reference. Otherwise the task is pushed onto the singly-linked oom_reaper_list under oom_reaper_lock and the kthread is woken.

oom_reaper() is a trivial loop: set_freezable(), then forever wait_event_freezable(oom_reaper_wait, oom_reaper_list != NULL), pop one task, call oom_reap_task().

oom_reap_task() retries oom_reap_task_mm() up to MAX_OOM_REAP_RETRIES (10) times with schedule_timeout_idle(HZ/10) — 100 ms — between attempts, so roughly one second of effort. On give-up it logs oom_reaper: unable to reap pid:%d (%s) and dumps sched_show_task() plus debug_show_all_locks() so you can see what was holding the lock. Either way it ends with set_bit(MMF_OOM_SKIP, &mm->flags).

oom_reap_task_mm() takes mmap_read_trylock(mm) — a trylock, never a blocking acquire, because blocking is the exact failure it exists to avoid. It then re-checks MMF_OOM_SKIP under the read lock, because that check “must run under mmap_lock for reading because it serializes against the mmap_write_lock();mmap_write_unlock() cycle in exit_mmap().”

__oom_reap_task_mm() is where pages are actually freed:

1. set_bit(MMF_UNSTABLE, &mm->flags) — tells everyone that the contents of this address space are no longer trustworthy. check_stable_address_space() in include/linux/oom.h tests this bit and returns VM_FAULT_SIGBUS, so any racing fault on a non-shared mapping gets a SIGBUS rather than silently reading zeroes where data used to be. The kernel-doc is explicit that the alternative is “memory corruption (zero pages instead of the original content)”.
2. Iterate every VMA. Skip VM_HUGETLB and VM_PFNMAP (hugetlb pages are not on the ordinary path; PFN maps are device memory the reaper does not own).
3. Reap a VMA if vma_is_anonymous(vma) || !(vma->vm_flags & VM_SHARED) — that is, anonymous mappings and private mappings of any kind, including private file mappings; only shared non-anonymous mappings are skipped. The comment gives the reason for the exclusion: “We do not even care about fs backed pages because all which are reclaimable have already been reclaimed and we do not want to block exit_mmap by keeping mm ref count elevated without a good reason.”
4. For each reapable VMA: mmu_notifier_range_init(), tlb_gather_mmu(), mmu_notifier_invalidate_range_start_nonblock() — and if that fails (a notifier, e.g. a GPU or KVM shadow-MMU listener, would have had to block) the VMA is skipped and the whole call returns false so the task is retried later. Otherwise unmap_page_range() tears down the PTEs and tlb_finish_mmu() flushes, returning the pages to the buddy allocator.

On success it logs, in the exact format you will see in dmesg:

oom_reaper: reaped process 3294284 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

process_mrelease(2) — the reaper as a syscall

Since Linux 5.15 (existence-checked: absent at v5.14, present at v5.15), userspace can invoke the same machinery directly:

SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags)

It resolves the pidfd, verifies task_will_free_mem() (the target must already be dying — you cannot use this to reap a live process), takes mmap_read_lock_killable(), and calls the very same __oom_reap_task_mm(). This is the missing piece for userspace OOM daemons: systemd-oomd, Android’s lmkd, and similar tools can SIGKILL a cgroup and then immediately force its memory back rather than waiting on the kernel’s two-second timer and the victim’s own scheduling. Note the difference from the kthread path: it uses a killable blocking read lock rather than a trylock, because the caller is a normal task that can be interrupted.

Configuration and Observability

vm.panic_on_oom

Three values, and the distinction between 1 and 2 is exactly the constraint classification from gate 5.

Value	CONSTRAINT_NONE (whole machine)	cpuset / mempolicy / memcg constrained	SysRq-triggered
0 (default)	kill a process	kill a process	kill a process
1	panic("Out of memory: system-wide panic_on_oom is enabled")	kill a process — other nodes may still have free memory, “system total status may be not fatal yet”	kill a process
2	panic("Out of memory: compulsory panic_on_oom is enabled")	panic — “Even oom happens under memory cgroup, the whole system panics”	kill a process

check_panic_on_oom() short-circuits on if (likely(!sysctl_panic_on_oom)) return;, so the default costs nothing. The sysctl is registered with .extra1 = SYSCTL_ZERO, .extra2 = SYSCTL_TWO, so writing 3 gets -EINVAL. is_sysrq_oom() is exempt in all cases — a manual echo f > /proc/sysrq-trigger should demonstrate the killer, not halt the box.

Values 1 and 2 are, per vm.rst, “for failover of clustering”: a node that panics reboots cleanly and its work fails over, which for an HA cluster beats limping along in an unknown state. The doc adds the forensics angle: “panic_on_oom=2 + kdump gives you very strong tool to investigate why oom happens. You can get snapshot.”

The other two sysctls

Sysctl	Default	Effect
vm.oom_dump_tasks	1 (enabled)	Emit the full per-task table on every kill. Disable only on machines with thousands of tasks where the dump itself is a problem — you are trading away the single best forensic artifact.
vm.oom_kill_allocating_task	0	Kill current instead of scanning. Avoids “the expensive tasklist scan”. Used on some embedded and real-time systems where a scan of every task under rcu_read_lock() is an unacceptable latency spike.

All three are registered from oom_init() in mm/oom_kill.c via register_sysctl_init("vm", vm_oom_kill_table), in the same subsys_initcall that starts the reaper thread with kthread_run(oom_reaper, NULL, "oom_reaper").

Reading a Real OOM Report, Field by Field

This is the artifact operators actually face, so it deserves a full decode. What follows is a genuine capture from the vault owner’s own workstation — a Framework Desktop with 128 GB of RAM running Fedora kernel 7.1.8-200.fc44, on 2026-08-29. Every printk format string quoted below was cross-checked against the v6.12 source; they are unchanged between the two versions.

1. The header — who asked, and for what

pipewire-pulse invoked oom-killer: gfp_mask=0x140dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO|__GFP_COMP), order=0, oom_score_adj=200

From dump_header(): "%s invoked oom-killer: gfp_mask=%#x(%pGg), order=%d, oom_score_adj=%hd". Four fields:

• pipewire-pulse is the invoker — current->comm. An audio daemon. It is not the problem; it simply needed a page at the wrong moment. Reading this name as the culprit is the single most common misreading of an OOM report.
• gfp_mask=0x140dca is the failing allocation’s GFP mask, printed raw and then symbolically by the %pGg printk specifier. The hex decodes exactly against the bit positions in include/linux/gfp_types.h: GFP_HIGHUSER_MOVABLE is 0x100cca, __GFP_ZERO is 0x100, __GFP_COMP is 0x40000 — summing to 0x140dca. GFP_HIGHUSER_MOVABLE is the mask for ordinary userspace pages, so this was a user page fault, not a kernel data structure. Because the mask contains __GFP_FS, gate 4 did not exempt it.
• order=0 — a single page. Nothing about fragmentation; the machine simply had no free pages left. (Had this been order=4 we would be looking at a fragmentation problem and the killer would have been exempted entirely.)
• oom_score_adj=200 is the invoker’s adjustment, not the victim’s. On a systemd desktop, everything under user@1000.service inherits +200 from the user slice.

Immediately after the header, dump_stack() prints the call chain, which in this capture is the clearest possible statement of the mental model:

 dump_header+0x43/0x1b3
 oom_kill_process.cold+0xa/0xba
 out_of_memory+0xfd/0x2f0
 __alloc_pages_slowpath.constprop.0+0x91f/0xb70
 __alloc_frozen_pages_noprof+0x31e/0x380
 alloc_pages_mpol+0xb2/0x180
 vma_alloc_folio_noprof+0x6a/0xd0
 alloc_anon_folio+0x1f0/0x460
 do_anonymous_page+0xf7/0x530
 __handle_mm_fault+0x47c/0x6c0
 handle_mm_fault+0x114/0x340
 exc_page_fault+0x90/0x1f0

Read bottom-up: a page fault on an anonymous mapping (Demand Paging) called the allocator, the fast path missed, the slow path exhausted itself, and out_of_memory() was called on this task’s stack.

2. Mem-Info — was this really exhaustion?

dump_header() calls __show_mem() (in mm/show_mem.c) for global OOMs. The lines that matter:

Node 0 active_anon:64281580kB inactive_anon:48085840kB active_file:26304kB inactive_file:77424kB
       unevictable:982364kB ... pagetables:574192kB ... all_unreclaimable? yes
Node 0 Normal free:344100kB boost:385024kB min:451700kB low:580996kB high:710292kB
       present:131564288kB managed:129306412kB
Node 0 Normal: 1707*4kB (UE) 3914*8kB (UE) 2994*16kB (UE) 827*32kB (UE) 1025*64kB (UE)
       1039*128kB (UE) 106*256kB (UE) 11*512kB (U) 0*1024kB 0*2048kB 0*4096kB = 343868kB
Free swap  = 128kB
Total swap = 8388604kB
33368657 pages RAM
581097 pages reserved

Five readings, in order of diagnostic value:

1. all_unreclaimable? yes — the decisive field. The node has nothing left that reclaim can take. If this said no, you would be looking at a premature or constrained OOM, not genuine exhaustion.
2. free:344100kB against min:451700kB — free memory is below the min watermark. Every allocation is failing the watermark check. See Watermarks and the Allocation Fast Path for how min/low/high are derived and what boost:385024kB (watermark boosting, which temporarily raises the bar after a fragmentation event) does to them.
3. active_file:26304kB inactive_file:77424kB — a hundred megabytes of page cache on a 128 GB box. Reclaim had already evicted essentially the entire cache. Compare active_anon:64281580kB — 64 GB of anonymous memory that reclaim cannot drop without swap.
4. Free swap = 128kB of Total swap = 8388604kB — swap is 100% consumed. With no swap left and 112 GB of anonymous memory, reclaim genuinely had no options.
5. The buddy free-list breakdown ends 0*1024kB 0*2048kB 0*4096kB — not one free block of order 8 or above. Even though order=0 here, this line is how you distinguish “no memory” from “no contiguous memory” on a high-order failure.

Those last two numbers also let you reconstruct the scoring denominator by hand. %lu pages RAM prints the sum of zone->present_pages, and %lu pages reserved the sum of present - managed; totalram_pages() is the managed total, so 33,368,657 − 581,097 = 32,787,560 pages. Swap contributes 8,388,604 kB / 4 = 2,097,151 pages. Therefore oc->totalpages = 34,884,711, and the oom_score_adj scale factor totalpages / 1000 is 34,884 pages per unit — about 136 MB per point of adjustment.

3. Tasks state — the per-task table

With vm.oom_dump_tasks=1 (the default), dump_tasks() prints a row per eligible task. Columns come straight from the pr_info in dump_task():

[  pid  ]   uid  tgid total_vm      rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
[   1881]     0  1881    10665     2298     1421      877         0   102400      405         -1000 systemd-udevd
[   2059]   998  2059     4008     1059      279      780         0    77824        0          -900 systemd-oomd
[   2552]     0  2552   972754     7210     6753      452         5   659456       94          -999 containerd
[   2766]     0  2766  1393054    60002    59304      698         0  1445888       12          -500 dockerd
[   4695]  1000  4695 381260286   240764   234571      781      5412  7843840    15973           300 brave
[3140489]  1000 3140489 21070721  1275770  1267539      855      7376 67919872        0           200 WARNO.exe
[3294284]  1000 3294284  9812676  9753282  9752742      540         0 78254080        0           200 python3
[3295074]  1000 3295074  9692813  9633273  9632733      540         0 77287424        0           200 python3

(eight rows shown; the real capture printed 403 of them — every eligible task on the machine)

All memory columns except pgtables_bytes are in pages; pgtables_bytes is in bytes. This is a real trap when eyeballing the table — brave shows total_vm 381260286 pages, which is 1.4 TB of virtual address space, and rss 240764 pages, which is 940 MB actually resident. total_vm is not scored and is nearly meaningless for a modern browser or JVM.

Now apply oom_badness() by hand, using the 34,884-pages-per-adjustment-unit factor derived above:

Task	rss	pgtables/4096	swapents	adj × 34,884	badness
python3 pid 3294284	9,753,282	19,105	0	+6,976,800	16,749,187 ← chosen
python3 pid 3295074	9,633,273	18,869	0	+6,976,800	16,628,942
brave pid 4695	240,764	1,915	15,973	+10,465,200	10,723,852
WARNO.exe pid 3140489	1,275,770	16,582	0	+6,976,800	8,269,152
containerd pid 2552	7,210	161	94	−34,849,116	−34,841,651 — negative, effectively immune
systemd-udevd pid 1881	—	—	—	adj == -1000	LONG_MIN — excluded outright

The scan picked the right process: two near-identical 39 GB Python jobs were running and the marginally larger one lost by about 120,000 pages. But look at rows three and four. A 940 MB browser tab at oom_score_adj=+300 outscores a 5 GB game at +200 by 2.4 million points, despite using a quarter of the memory. That is the oom_score_adj scale factor talking: on this machine each adjustment point is worth 136 MB of pretend RSS, so a 100-point difference in policy outweighs a 4 GB difference in reality. This is precisely Michal Hocko’s complaint that the knob is unusable for fine tuning — it is a policy override, not a weighting.

4. The verdict lines

oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,
  task_memcg=/user.slice/user-1000.slice/user@1000.service/app.slice/app-ghostty-surface-transient-4142.scope,
  task=python3,pid=3294284,uid=1000
Out of memory: Killed process 3294284 (python3) total-vm:39250704kB, anon-rss:39010968kB,
  file-rss:2160kB, shmem-rss:0kB, UID:1000 pgtables:76420kB oom_score_adj:200
oom_reaper: reaped process 3294284 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

• constraint=CONSTRAINT_NONE — genuine machine-wide exhaustion. Had it read CONSTRAINT_CPUSET or CONSTRAINT_MEMORY_POLICY, the right response would be to look at NUMA placement, not at total memory. nodemask=(null) because out_of_memory() clears oc->nodemask for any constraint other than CONSTRAINT_MEMORY_POLICY.
• global_oom and task_memcg=... come from mem_cgroup_print_oom_context(). Even for a global OOM the victim’s cgroup is printed, which on a systemd machine is the fastest way to identify which application it belonged to — here, a shell scope under a terminal emulator.
• The kill line is __oom_kill_process()’s pr_err. anon-rss:39010968kB is 37 GB of anonymous memory in one process. pgtables:76420kB — 74 MB of page tables, matching the 78254080 bytes in the task table.
• oom_reaper: reaped ... two seconds later (log timestamps 00:10:16 → 00:10:18) is the OOM_REAPER_DELAY timer firing exactly on schedule. now anon-rss:0kB means the reaper got the read lock on the first try and unmapped everything.

5. A memcg OOM for contrast

The same machine, three days later, inside a Docker container:

pytest invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
memory: usage 2097152kB, limit 2097152kB, failcnt 2305
Memory cgroup stats for /system.slice/docker-<id>.scope: ...
oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-<id>.scope,mems_allowed=0,
  oom_memcg=/system.slice/docker-<id>.scope,task_memcg=/system.slice/docker-<id>.scope,task=pytest,pid=764944
Memory cgroup out of memory: Killed process 764944 (pytest) total-vm:3966812kB, anon-rss:2078976kB, ...

Four differences worth internalising, all traceable to is_memcg_oom(oc) branches in the source:

	Global OOM	memcg OOM
Constraint	CONSTRAINT_NONE	CONSTRAINT_MEMCG, plus an oom_memcg= field naming the OOMing cgroup
Memory dump	__show_mem() — full Mem-Info, zone watermarks, buddy free lists	mem_cgroup_print_oom_meminfo() — memory: usage/limit/failcnt and Memory cgroup stats. No zone or watermark data, because zones are irrelevant
Task table	every process on the machine	only tasks in the OOMing cgroup (mem_cgroup_scan_tasks)
Notifier / kill message	notifier chain consulted; "Out of memory"	notifier chain skipped (if (!is_memcg_oom(oc))); "Memory cgroup out of memory"

Note also gfp_mask=0xcc0(GFP_KERNEL) — the exact composite value derived in GFP Flags and Allocation Contexts — and that no oom_reaper line follows: the victim exited on its own inside the two-second window, so wake_oom_reaper() saw MMF_OOM_SKIP and did nothing. That is the healthy case.

The failcnt 2305 is the accumulated count of times this cgroup hit its limit — evidence that the container had been bouncing off memory.max for a while before the fatal one. See The Memory Cgroup memcg for the charging path and cgroup OOM and memory.oom.group for the memcg OOM sequence in detail.

Failure Modes

Thrashing on the edge — the killer that never fires. This is the most common and most damaging failure, and it is a failure of omission. When the working set marginally exceeds RAM, reclaim keeps almost succeeding: pages are evicted, faulted back in, evicted again. Free memory never quite reaches zero, so should_reclaim_retry() keeps returning true and out_of_memory() is never called. The machine is alive, responsive to ping, and completely useless — every task spends its time in direct reclaim and swap I/O. The kernel has no notion of “this is taking too long”; it only knows “memory is technically still being reclaimed.” Pressure Stall Information exists precisely to expose this: /proc/pressure/memory’s full avg10 climbing toward 100 means every task is stalled on memory, which is the signal the OOM killer does not have. Facebook’s oomd README puts a number on the cost: after deploying PSI-driven userspace killing, “we’ve regularly seen 30 minute host lockups go away entirely.”

Over-protection turning a kill into a panic. oom_score_adj = -1000 is absolute. If every remaining candidate is protected, select_bad_process() returns nothing and gate 9 calls panic("System is deadlocked on memory"). The rule of thumb is to reserve -1000 for the handful of processes whose death is worse than a reboot, and use -900/-500 — which reduce but do not eliminate candidacy — for everything else. The real system captured above does exactly this: systemd-udevd and auditd at -1000, containerd at -999, systemd-oomd/snapd/dbus-broker at -900, dockerd at -500, systemd-journald at -250.

oom_reaper: unable to reap. Ten failed mmap_read_trylock() attempts over roughly one second. The mmap_lock is held for writing by a task that is itself wedged. Memory still returns eventually — when the victim finally exits — but the guarantee the reaper provides is gone, and under sustained pressure the original livelock reopens. The log includes sched_show_task() and debug_show_all_locks() for the victim precisely so you can identify the write-side holder.

Killing the wrong process. oom_badness() scores get_mm_rss(), which includes file and shmem pages. A process that mmaps and touches a large file has a large RSS without being a leak; it can out-score the actual leaker. Similarly, a workload split into 100 processes of 1 GB each will never be chosen over a single 40 GB process, no matter which is the real problem — a limitation Roman Gushchin’s 2018 cgroup-aware OOM killer set out to fix, and which was only partially resolved (see below).

GFP_NOFS allocations that quietly never kill. Gate 4 means a filesystem hot path allocating under memalloc_nofs_save() will loop rather than trigger a kill. If a workload wedges with high sys time in xfs_/ext4_ writeback and no OOM report ever appears, this is a candidate explanation. The fix is not to remove the scope guard — that reintroduces the recursion deadlock — but to reduce the allocation footprint inside it. See GFP Flags and Allocation Contexts.

The invoker-is-the-victim confusion. Repeated because it costs the most time in incident review: the process in the first line of the report is current, the process in the last line is the victim. They are almost never the same, unless vm.oom_kill_allocating_task is set.

Alternatives — Front-Running the Kernel with PSI

The consensus among memory-management developers and large-scale operators is that the kernel OOM killer, as a policy engine, is the wrong tool. Not because its implementation is bad but because of where it sits: it is invoked only when an allocation is about to fail, which is minutes of thrashing after the machine stopped being useful. The modern architecture is to keep the kernel killer as an unconditional backstop and put a PSI-driven userspace daemon in front of it.

flowchart TB
    subgraph HEALTHY["healthy"]
        H1["free pages above the low watermark<br/>allocations take the fast path"]
    end
    subgraph PRESSURE["pressure — minutes"]
        P1["kswapd reclaiming continuously"]
        P2["direct reclaim stalls in allocating tasks"]
        P3["PSI memory.full avg10 climbing<br/>toward 100 percent"]
    end
    subgraph USERSPACE["userspace tier — acts here"]
        U1["systemd-oomd<br/>PSI over 60 percent for 30 s,<br/>or swap over 90 percent used"]
        U2["Facebook oomd<br/>PSI plus plugin policy"]
        U3["earlyoom<br/>MemAvailable and free swap both under 10 percent"]
    end
    subgraph KERNEL["kernel tier — the floor"]
        K1["allocation actually fails the watermark"]
        K2["out_of_memory: pick the largest RSS, SIGKILL"]
    end
    H1 --> P1 --> P2 --> P3
    P3 -->|"seconds to minutes of warning"| U1
    P3 --> U2
    P3 --> U3
    U1 -->|"SIGKILL whole cgroup"| DONE["pressure relieved<br/>before any allocation fails"]
    U2 --> DONE
    U3 --> DONE
    P3 -->|"userspace absent, starved,<br/>or too slow"| K1 --> K2

The two-tier defence. What it shows: PSI turns a binary condition (“did an allocation fail?”) into a continuous one (“how much time are tasks losing to memory?”), which gives userspace a window of seconds to minutes to act. The insight: the userspace tier is a policy layer and the kernel tier is a safety layer, and you keep both. A userspace daemon can be starved of CPU or memory itself and fail to run; the kernel killer runs on the stack of whoever is allocating, so it cannot be starved out.

	systemd-oomd	Facebook oomd	earlyoom	kernel OOM killer
Signal used	cgroup v2 PSI + swap usage	PSI + arbitrary metrics via plugins	MemAvailable and free swap percentages	watermark failure after reclaim exhaustion
Requires	cgroup v2, PSI (Linux 4.20+), memory accounting on monitored units	PSI (4.20+), cgroup v2	nothing — pure /proc polling	nothing
Granularity	a whole cgroup	configurable via plugins	one process	one process (plus mm sharers, plus memory.oom.group subtree)
Signal sent	SIGKILL to every process in the chosen cgroup	SIGKILL by default; plugins can do anything (RPC backpressure, log dumps)	SIGTERM — graceful	SIGKILL
Default thresholds	DefaultMemoryPressureLimit=60% sustained for DefaultMemoryPressureDurationSec=30s; SwapUsedLimit=90%	policy-defined	both available memory and free swap under 10%	n/a
Policy flexibility	per-unit ManagedOOMMemoryPressureLimit=, ManagedOOMSwap=	C++ plugin system	a few CLI flags and a process-name avoid/prefer regex	oom_score_adj only

Detail worth knowing about each, from their own documentation:

• systemd-oomd (man source at v261.2) “uses cgroups-v2 and pressure stall information (PSI) to monitor and take corrective action before an OOM occurs in the kernel space.” Its candidate rules are specific and often surprise people: only descendant cgroups are candidates — the unit whose ManagedOOMMemoryPressure=kill property you set is itself immune — and only leaf cgroups or cgroups with memory.oom.group=1 are eligible. Its “memory pressure” is the PSI full metric: “the fraction of time in a 10 second window in which all tasks in the control group were delayed.” It also warns loudly that swap is not optional: “Without swap, the system enters a livelocked state much more quickly and may prevent systemd-oomd from responding in a reasonable amount of time.”
• Facebook’s oomd (README) is the daemon that motivated PSI’s upstreaming, open-sourced under GPLv2 in July 2018 (LWN). Its framing of why kernel-side OOM handling is structurally hard is worth quoting: the kernel “can spend an unbounded amount of time swapping in and out pages and evicting the page cache. Furthermore, configuring policy is not very flexible while being somewhat complicated.” Its plugin model exists so a service can respond with something other than death — sending a back-off RPC upstream, or dumping diagnostics to a remote service, before anything is killed.
• earlyoom (README) is the minimal option: pure C, no dependencies, no PSI, no cgroups. It polls up to ten times a second and acts when both available memory and free swap fall below 10%. Two design choices distinguish it. It watches MemAvailable, not MemFree, and explains why: “On a healthy Linux system, ‘free’ memory is supposed to be close to zero, because Linux uses all available physical memory to cache disk access.” And it sends SIGTERM, giving the target a chance to shut down cleanly — something the kernel killer, which must guarantee the memory comes back, cannot afford to do.

What the kernel tried, and what it kept

The kernel side has not stood still, but very little of the ambitious work landed.

• Cgroup-aware victim selection (2018, Roman Gushchin). The proposal was to find the control group with the largest memory consumption and kill the largest process within it — fixing the “one big process versus a hundred small ones” blind spot. It went into the -mm tree for 4.19 and was heavily contested; David Rientjes objected that root-cgroup processes were scored differently, that the comparison was not hierarchical (so a workload could evade selection simply by splitting itself across subgroups), and that oom_score_adj was ignored inside cgroups (LWN, 2018-07-27). Checking the outcome against v6.12: only the group-kill switch survived. select_bad_process() is still strictly per-task, there is no cgroup-size comparison anywhere in mm/oom_kill.c, and the memory.oom_policy knob Rientjes proposed does not exist. What shipped is memory.oom.group — note the merged name uses dots, not the underscore used in the 2018 discussion.
• BPF-programmable victim selection (2023–, Chuyi Zhou and successors). The idea is to replace the oom_badness() call with a BPF hook — int bpf_oom_evaluate_task(struct task_struct *task, struct oom_control *oc) returning one of NO_BPF_POLICY / BPF_EVAL_ABORT / BPF_EVAL_NEXT / BPF_EVAL_SELECT (LWN, 2023-08-17). Not present in v6.12: grep -c bpf mm/oom_kill.c at the v6.12 tag returns 0. Treat this as an active area to re-check rather than an available feature.

Uncertain

Verify: whether any BPF-based OOM policy hook has been merged in the 6.13–7.1 window. Reason: the design has been reposted several times since 2023 and LWN was covering it again in 2025 (article 1019230, “Custom out-of-memory killers in BPF”), but lwn.net returned HTTP 429 for that article on every attempt during this research and lore.kernel.org is now behind a proof-of-work bot check that curl cannot clear (HTTP 200 with a “Making sure you’re not a bot!” interstitial). What is verified: no BPF hook exists in mm/oom_kill.c at v6.12. To resolve: grep -i bpf mm/oom_kill.c at the v7.1 tag, or read Documentation/admin-guide/mm/ for a bpf_oom entry. uncertain

Production Notes

In containers you meet the cgroup killer first, but the global one still exists. A container almost always hits its own memory.max and takes a CONSTRAINT_MEMCG kill long before the host runs dry — that is the second capture above, and it is the overwhelmingly common case in Kubernetes. But three things still drive a host into CONSTRAINT_NONE: an unconstrained system-slice process leaking (the kubelet, a monitoring agent, a log shipper), aggregate overcommit across many containers whose limits sum above physical RAM, and kernel memory that is not charged to any container. When that happens the global heuristic chooses among everything on the box, including your control plane. Hence the near-universal practice of setting strongly negative oom_score_adj on the container runtime, sshd, and the node agent — visible in the real capture as containerd at -999 and dockerd at -500.

Running a userspace OOM daemon does not make the kernel killer stop firing. In the captured incident systemd-oomd was running (PID 2059, oom_score_adj -900) and the kernel killer fired anyway. Two workloads allocated 39 GB each in a burst; the PSI threshold requires pressure sustained for 30 seconds by default, and the allocation rate outran it. Userspace tiering reduces the frequency of kernel kills dramatically; it does not eliminate them, and tuning DefaultMemoryPressureDurationSec down trades false positives for coverage.

Collect the report, not just the fact of the kill. The habit that resolves incidents fastest:

journalctl -k --since "-1h" | grep -E 'invoked oom-killer|oom-kill:|Killed process|oom_reaper'

then go back and read the whole block for the timestamp you find. Specifically check, in order: constraint= (is this really machine-wide?), all_unreclaimable? (was reclaim genuinely out of options?), Free swap (was swap exhausted, or unconfigured?), and the task table sorted by rss (was the victim actually the largest, or did oom_score_adj pick it?). If you keep PSI history, overlay /proc/pressure/memory for the preceding minutes — the shape of the pressure curve distinguishes a sudden allocation burst (near-vertical) from a slow leak (a long ramp), and those have completely different fixes.

Do not disable vm.oom_dump_tasks unless you have measured the cost. The table is the only record of what the machine looked like at the instant of death. On a host with a few hundred tasks it costs milliseconds.

Test your protection, because -1000 is load-bearing. echo f > /proc/sysrq-trigger triggers a SysRq OOM — exempt from panic_on_oom, exempt from the “victim already in flight” abort — which is a safe way to confirm that your oom_score_adj policy chooses the process you expect on a staging host.

See Also

• OOM Score and Victim Selection — the history of the oom_badness() heuristic and the selection walk in more depth (this note covers when and how the kill happens; that note covers whom, and why the heuristic looks the way it does).
• cgroup OOM and memory.oom.group — the per-cgroup OOM path, memory.max exhaustion, memory.oom.group, and the cgroup-v1 memory.oom_control user-notification interface.
• The Memory Cgroup memcg — the charging path and per-cgroup reclaim that precede a memcg OOM.
• Memory Reclaim Overview — the whole reclaim hierarchy whose exhaustion is a precondition for every OOM kill.
• Direct Reclaim — the sleeping reclaim path in the allocator slow path, immediately upstream of __alloc_pages_may_oom().
• kswapd and Background Reclaim — the background thread the slow path wakes before reclaiming itself.
• Watermarks and the Allocation Fast Path — min/low/high, watermark boosting, and how ALLOC_OOM lets a victim dip below min to finish dying.
• GFP Flags and Allocation Contexts — why GFP_ATOMIC never triggers OOM, what __GFP_FS gates at gate 4, and how to decode the gfp_mask= field of an OOM report.
• Memory Compaction — what actually helps high-order allocations, which the OOM killer explicitly refuses to.
• Memory Overcommit and Accounting — vm.overcommit_memory, which decides whether allocations succeed optimistically and thus whether OOM is reachable at all.
• Pressure Stall Information — the signal every modern userspace OOM daemon consumes.
• Anonymous vs File-Backed Memory — why the reaper targets anonymous and private mappings.
• The Buddy Allocator — where reaped pages go back to.
• MOC: Linux Memory Management MOC (§11, The Out-of-Memory Killer).