Direct Reclaim

Direct reclaim is the kernel’s synchronous, last-ditch way to find free memory: when a task calls into the page allocator, the allocator cannot satisfy the request even against its lowest watermark, and the request is one that is allowed to block, the allocating task itself descends into the reclaim machinery — running try_to_free_pages() in its own context — to free pages before retrying the allocation. It is the opposite of background reclaim: instead of a dedicated kernel thread quietly freeing memory ahead of demand, the thread that wanted the memory is forced to stop and do the janitorial work first. Because the allocation blocks until reclaim makes progress, direct reclaim shows up directly as allocation latency — a stall in the application’s own timeline — which is why a high direct-reclaim rate (the allocstall / pgscan_direct counters) is one of the clearest signals that a machine is genuinely short of memory. This note covers the synchronous path in the allocator slow path; the asynchronous counterpart lives in kswapd and Background Reclaim and the shared page-scanning machinery in Memory Reclaim Overview and The LRU Lists.

Version pin — Linux 6.12 LTS, verified 2026-09-04

Every function name, constant, and line reference below was read from the v6.12 tag of torvalds/linux via raw.githubusercontent.com. 6.12 is a maintained longterm branch, not mainline: https://www.kernel.org/releases.json on 2026-09-04 lists mainline at 7.3-rc1, latest stable at 7.2.3, and 6.12.108 among the longterm entries — which is why 6.12 is the pin rather than the newest tag. Claims were cross-checked against the 6.18 LTS tag (6.18.49, also longterm), and anything that changed after 6.12 is dated inline against the release that changed it.

Two corrections to an earlier revision of this note, both verified by diffing the two tags: try_to_free_pages() and throttle_direct_reclaim() are byte-identical between 6.12 and 6.18, but pgdat_balanced() is not — 6.18 adds defrag_mode handling and a percpu_drift_mark snapshot — and pgdat->kswapd_failures changed from a plain int to an atomic_t. See kswapd and Background Reclaim, which owns that function.

Scope — where this note sits among its siblings

Reclaim is documented here as a three-way split, and each note is authoritative for its own third:

• Memory Reclaim Overview is the map — what reclaim is, the watermark arithmetic, what counts as reclaimable, how victims are chosen off the LRU or by MGLRU, shrinkers, and a worked /proc/vmstat diagnosis of one measured machine.
• kswapd and Background Reclaim owns the daemon — the per-node kthread, its sleep/wake state machine, balance_pgdat(), watermark boosting, the kcompactd handoff, and the counters that describe kswapd’s own health.
• This note owns the stall — how a task enters reclaim from __alloc_pages_slowpath(), what try_to_free_pages() does in that task’s own context, the two throttles that can put it to sleep, how the stall is accounted (allocstall, pgscan_direct, PSI), and — the practical heart — how to recognise a direct-reclaim latency stall in production.

The page-scanning engine itself (shrink_node() → shrink_lruvec() → the LRU walk) is shared by both reclaimers and is deliberately explained in neither of the two trigger notes; see Memory Reclaim Overview and The LRU Lists.

Mental Model — Reclaim in the Caller’s Own Hands

The page allocator has a fast path and a slow path. The fast path (Watermarks and the Allocation Fast Path) tries get_page_from_freelist() against the low watermark (ALLOC_WMARK_LOW) and, if a zone has enough free pages above that line, hands a page back with no further ceremony. When the fast path fails, control falls into __alloc_pages_slowpath(), and that function is where direct reclaim lives. The slow path does a specific, ordered set of escalations, and direct reclaim is deliberately not the first of them.

flowchart TD
  REQ["__alloc_pages()<br/>fast path: get_page_from_freelist<br/>(ALLOC_WMARK_LOW)"] -->|"fail"| SLOW["__alloc_pages_slowpath()"]
  SLOW --> WAKE["wake_all_kswapds()<br/>(if __GFP_KSWAPD_RECLAIM)"]
  WAKE --> RETRY1["retry get_page_from_freelist<br/>(ALLOC_WMARK_MIN)"]
  RETRY1 -->|"success"| DONE["return page"]
  RETRY1 -->|"fail"| CANBLOCK{"can_direct_reclaim?<br/>(__GFP_DIRECT_RECLAIM set,<br/>not PF_MEMALLOC)"}
  CANBLOCK -->|"no"| NOPAGE["fail / NULL<br/>(atomic alloc)"]
  CANBLOCK -->|"yes"| DR["__alloc_pages_direct_reclaim()<br/>= psi_memstall_enter +<br/>__perform_reclaim()"]
  DR --> TTFP["try_to_free_pages()<br/>do_try_to_free_pages → shrink_zones<br/>(ALLOCSTALL counted here)"]
  TTFP -->|"freed some"| RETRY2["get_page_from_freelist again"]
  RETRY2 -->|"success"| DONE
  RETRY2 -->|"still fail"| COMPACT["direct compaction,<br/>then retry / OOM"]

The allocator slow path in Linux 6.12. What it shows: kswapd is woken first (wake_all_kswapds), the allocator retries against the lower min watermark, and only if that still fails — and the request is allowed to block — does the task enter __alloc_pages_direct_reclaim() and run try_to_free_pages() synchronously. The insight to take: by the time a task is in direct reclaim, the per-node kswapd has already been asked to help and could not free pages fast enough. Direct reclaim is therefore by construction a sign that background reclaim has fallen behind demand — it is the allocator’s admission that it must block the caller to survive.

The single most important idea: direct reclaim is reclaim performed in the context of, and at the expense of, the task that wanted the memory. There is no separate thread; current — your malloc(), your page-fault handler, your read() populating the page cache — runs try_to_free_pages() on its own stack and does not return from the allocation until that function either freed enough pages or gave up. That is the whole reason it is a latency problem.

Why the mechanism exists at all

It helps to know that direct reclaim is not a bolt-on: it is the older of the two paths in spirit, and background reclaim was layered over it to hide it. Mel Gorman’s Understanding the Linux Virtual Memory Manager describes the 2.4-era arrangement plainly: kswapd “keeps freeing pages until the pages_high watermark is reached,” and “under extreme memory pressure, processes will do the work of kswapd synchronously” by calling balance_classzone() → try_to_free_pages_zone() (Ch. 10, Page Frame Reclamation). Twenty years and a rename later the sentence is still exactly true of 6.12; only the function names moved. The design has always been two actors running one engine, with the synchronous actor as the backstop that guarantees forward progress when the asynchronous one cannot be trusted to have finished in time.

That framing explains an asymmetry that surprises people the first time they measure it: direct reclaim is not merely later than background reclaim, it is structurally worse at its job. The two run identical code, but they run it against different worlds.

	kswapd (background)	Direct reclaim (synchronous)
Who runs it	Per-node kswapd<N> kthread	The allocating task itself (current)
Entry condition	Some zone fell below WMARK_LOW	The WMARK_MIN retry in the slowpath failed
Target	Σ max(high_wmark_pages(zone), 32) over eligible zones (kswapd_shrink_node)	SWAP_CLUSTER_MAX = 32 folios (try_to_free_pages)
Priority loop	DEF_PRIORITY (12) down to 1	DEF_PRIORITY (12) down to 0
State of the LRU when it starts	Freshly below low; cheap clean file pages still abundant	Already picked over by kswapd; what is left is dirty, mapped, or under writeback
Latency charged to	Nobody’s critical path (a kthread’s CPU time)	The allocation, hence the syscall, hence the user request
Throttled by too_many_isolated()	No — current_is_kswapd() returns early	Yes
Throttled by VMSCAN_THROTTLE_CONGESTED	No — gated on !current_is_kswapd()	Yes
Throttled by VMSCAN_THROTTLE_NOPROGRESS	No — explicitly excluded	Yes, at priority == 1 with nothing reclaimed
Stalls on pfmemalloc_wait	No — it is the thing that wakes the sleepers	Yes, via throttle_direct_reclaim()
vmstat attribution	pgscan_kswapd / pgsteal_kswapd, pageoutrun	pgscan_direct / pgsteal_direct, allocstall_<zone>

The two reclaimers compared, every row read from mm/vmscan.c at v6.12. What it shows: the differences are not cosmetic. Direct reclaim asks for a far smaller batch (32 folios versus a whole zone’s worth), descends one priority level further, and is subject to four back-off mechanisms that kswapd is explicitly exempt from. The insight to take: every one of those exemptions exists because kswapd is trusted to be the thing that resolves the pressure, while a direct reclaimer is one of many competing tasks that must be prevented from stampeding. A direct reclaimer is therefore not just “kswapd, but on your thread” — it is a deliberately throttled, deliberately small-batch, deliberately deprioritised participant, and that is precisely why its measured scan-to-steal efficiency is so much worse (see the ratio worked out in Memory Reclaim Overview).

Mechanical Walk-through

How the slow path reaches direct reclaim

When get_page_from_freelist() fails on the fast path, __alloc_pages_slowpath() runs. Its first reclaim-related act, gated on the ALLOC_KSWAPD flag (set when the GFP mask includes __GFP_KSWAPD_RECLAIM), is:

if (alloc_flags & ALLOC_KSWAPD)
    wake_all_kswapds(order, gfp_mask, ac);

wake_all_kswapds() walks every zone in the allocation’s zonelist and calls wakeup_kswapd() on each distinct node’s pg_data_t — kicking the per-node background daemon (see kswapd and Background Reclaim). The slow path then retries get_page_from_freelist(), this time using the relaxed ALLOC_WMARK_MIN watermark (the min line is below low, so a little more of the reserve is now usable). If that succeeds, the task escapes without ever doing reclaim itself — kswapd, or simply the relaxed watermark, was enough.

Only when that retry also fails does the slow path consider direct reclaim. Two gates must be open:

/* Caller is not willing to reclaim, we can't balance anything */
if (!can_direct_reclaim)
    goto nopage;
 
/* Avoid recursion of direct reclaim */
if (current->flags & PF_MEMALLOC)
    goto nopage;

can_direct_reclaim is true only if the GFP mask has __GFP_DIRECT_RECLAIM set. An atomic allocation (GFP_ATOMIC, or any context that cannot sleep — interrupt handlers, code holding a spinlock) clears that bit; such an allocation simply fails rather than blocking. The second gate, PF_MEMALLOC, prevents recursion: a task already inside reclaim (or kswapd, which sets PF_MEMALLOC permanently — see kswapd and Background Reclaim) must not re-enter reclaim and risk an unbounded loop. See GFP Flags and Allocation Contexts for how the mask encodes “may I sleep / do I/O / trigger reclaim.”

One detail is easy to miss and matters enormously for the relationship between the two reclaimers: wake_all_kswapds() is called a second time, at the top of the retry: label, not just once on entry. The source comment says why in one line — “Ensure kswapd doesn’t accidentally go to sleep as long as we loop.” Every trip around the slowpath retry loop re-kicks every node’s daemon. A task that is looping in direct reclaim is therefore also continuously preventing kswapd from completing its two-stage sleep, which is exactly the mechanism behind the kswapd_low_wmark_hit_quickly counter climbing in lockstep with allocstall on a loaded machine (both counters are dissected in Memory Reclaim Overview).

The gates, in the order the slowpath evaluates them, decide whether a given allocation can ever stall at all:

Gate	Source	Effect when it fails
alloc_flags & ALLOC_KSWAPD	Derived from __GFP_KSWAPD_RECLAIM in gfp_to_alloc_flags()	kswapd is never woken; the allocation neither helps nor is helped by background reclaim
get_page_from_freelist() at ALLOC_WMARK_MIN	gfp_to_alloc_flags() starts at ALLOC_WMARK_MIN | ALLOC_CPUSET	Falls through toward reclaim; success here is the common escape
can_direct_reclaim	gfp_mask & __GFP_DIRECT_RECLAIM	goto nopage — the allocation returns NULL without ever sleeping
!(current->flags & PF_MEMALLOC)	Recursion guard	goto nopage — a reclaimer that needs memory is not allowed to re-enter reclaim
costly_order && (!can_compact || !__GFP_RETRY_MAYFAIL)	order > PAGE_ALLOC_COSTLY_ORDER (which is 3, mm/internal.h line 46)	One pass of reclaim and compaction, then goto nopage — no retry loop
should_reclaim_retry()	no_progress_loops > MAX_RECLAIM_RETRIES (16, mm/internal.h line 468)	Stop retrying reclaim; try compaction retries, then the OOM killer

The gate sequence in __alloc_pages_slowpath() at v6.12. What it shows: four separate conditions must all be permissive before a task is allowed to block, and a fifth (should_reclaim_retry) governs how many times it may block. The insight to take: whether an allocation stalls is decided almost entirely by the caller’s GFP flags and requested order, not by how much memory is free. Two allocations on the same machine at the same instant — one GFP_KERNEL order-0, one GFP_ATOMIC — take completely different paths through this table, which is why “is the machine stalling?” is never answerable from free-memory numbers alone.

should_reclaim_retry() deserves a closer look because it is the function that decides direct reclaim has failed. Its no_progress_loops counter is reset to zero whenever reclaim made progress and order <= PAGE_ALLOC_COSTLY_ORDER; for costly orders it is incremented unconditionally, with the source comment explaining that “costly allocations might have made a progress but this doesn’t mean their order will become available due to high fragmentation.” Once past 16 loops, or once no zone could satisfy the request even if every reclaimable page in it were freed — the check is __zone_watermark_ok(zone, order, min_wmark, ..., available) where available = zone_reclaimable_pages(zone) + NR_FREE_PAGES — the function returns false and the slowpath heads for the OOM killer. There is one last act before it gives up: unreserve_highatomic_pageblock(ac, true) releases the high-order atomic reserve, on the principle that starving an atomic allocator is better than killing a process.

__alloc_pages_direct_reclaim and the PSI stall window

With both gates open, the slow path calls:

page = __alloc_pages_direct_reclaim(gfp_mask, order, alloc_flags, ac,
                                    &did_some_progress);

This wrapper does three things. It brackets the work in Pressure Stall Information (PSI) accounting — psi_memstall_enter(&pflags) … psi_memstall_leave(&pflags) — so that the time the task spends blocked here is attributed to memory stall and surfaces in /proc/pressure/memory (see Pressure Stall Information). Between those it calls __perform_reclaim(), and afterward it retries get_page_from_freelist(); if that still fails it releases any high-order atomic reserves and drains the per-CPU page lists (unreserve_highatomic_pageblock() then drain_all_pages()) once before trying a final time. The PSI bracket is why direct reclaim is the canonical thing PSI was built to measure — it is, almost by definition, “an application stalled waiting on memory.”

flowchart TD
  ENTER["__alloc_pages_direct_reclaim()"]
  PSI1["psi_memstall_enter(&amp;pflags)<br/>THE STALL CLOCK STARTS"]
  PR["__perform_reclaim()"]
  CR1["cond_resched()"]
  CMP["cpuset_memory_pressure_bump()"]
  FSR["fs_reclaim_acquire(gfp_mask)<br/>(lockdep annotation only)"]
  NR["memalloc_noreclaim_save()<br/>sets PF_MEMALLOC on current"]
  TTFP["try_to_free_pages(zonelist, order,<br/>gfp_mask, nodemask)"]
  REST["memalloc_noreclaim_restore()<br/>fs_reclaim_release()<br/>cond_resched()"]
  PROG{"did_some_progress<br/>== 0 ?"}
  GPF["get_page_from_freelist()"]
  GOT{"got a page?"}
  DRAINED{"already drained<br/>this call?"}
  DRAIN["unreserve_highatomic_pageblock(ac,false)<br/>drain_all_pages(NULL)<br/>drained = true"]
  PSI2["psi_memstall_leave(&amp;pflags)<br/>THE STALL CLOCK STOPS"]
  OUT["return page (or NULL)"]

  ENTER --> PSI1 --> PR
  PR --> CR1 --> CMP --> FSR --> NR --> TTFP --> REST --> PROG
  PROG -->|"yes"| PSI2
  PROG -->|"no"| GPF
  GPF --> GOT
  GOT -->|"yes"| PSI2
  GOT -->|"no"| DRAINED
  DRAINED -->|"no"| DRAIN --> GPF
  DRAINED -->|"yes"| PSI2
  PSI2 --> OUT

The inside of one direct-reclaim episode, mm/page_alloc.c at v6.12. What it shows: the PSI bracket is the outermost thing in the function, so every microsecond spent here — including the freelist retry and the per-CPU drain, not just the scanning — is charged to memory pressure. Note also that PF_MEMALLOC is set on the task for the duration by memalloc_noreclaim_save(), which is the same flag the slowpath checked as its recursion gate a few lines earlier: while you are in reclaim, any allocation you make cannot itself reclaim. The insight to take: the drain-and-retry step exists because reclaim frees pages into the buddy allocator globally, and by the time you look, another CPU may have taken them or they may be sitting on some other CPU’s per-CPU list. drain_all_pages() is an IPI to every CPU asking it to flush those lists — expensive, which is why it happens at most once, and a real reason a single “stall” can cost far more than the scanning alone suggests.

The cpuset_memory_pressure_bump() call is a small piece of history worth naming: it feeds the per-cpuset memory_pressure file, a cpuset-v1 counter of direct-reclaim entries per second. It is dead weight on a cgroup-v2 machine, but it is the reason the very first thing __perform_reclaim() does after cond_resched() is bump a counter that most readers have never heard of.

Note what __perform_reclaim() does not do: fs_reclaim_acquire() is a pure lockdep annotation. It takes a fake lock so that if any filesystem code later tries to allocate with __GFP_FS while holding a lock that reclaim can itself wait on, lockdep reports the deadlock at development time rather than at 3 a.m. It has zero runtime cost on a production kernel with CONFIG_LOCKDEP=n, and it does not gate anything.

try_to_free_pages — the actual reclaim

__perform_reclaim() is thin. It calls cond_resched(), marks the task as being in fs_reclaim context (a lockdep aid that catches deadlocks where a filesystem allocation recurses into the same filesystem during writeback), sets memalloc_noreclaim to prevent the just-described recursion, and calls:

progress = try_to_free_pages(ac->zonelist, order, gfp_mask, ac->nodemask);

try_to_free_pages() builds a struct scan_control — the kernel’s per-reclaim “request object” — and hands it to do_try_to_free_pages(), the shared engine that both direct reclaim and kswapd funnel into. The initialiser is short enough to read whole, and every field in it is a policy decision:

struct scan_control sc = {
	.nr_to_reclaim = SWAP_CLUSTER_MAX,        /* 32 folios — that is the whole ask */
	.gfp_mask      = current_gfp_context(gfp_mask),
	.reclaim_idx   = gfp_zone(gfp_mask),      /* which allocstall_<zone> gets bumped */
	.order         = order,
	.nodemask      = nodemask,
	.priority      = DEF_PRIORITY,            /* 12 */
	.may_writepage = !laptop_mode,            /* NOT unconditionally 1 */
	.may_unmap     = 1,
	.may_swap      = 1,
};

Field by field: SWAP_CLUSTER_MAX is 32UL (include/linux/swap.h line 225) — a direct reclaimer asks for 32 folios and stops, which is a strikingly modest goal for something that can cost tens of milliseconds. reclaim_idx comes from gfp_zone(gfp_mask), and it is what selects which allocstall_<zone> counter is incremented; because GFP_HIGHUSER_MOVABLE maps to ZONE_MOVABLE, ordinary page-cache and anonymous allocations land in allocstall_movable even on machines whose ZONE_MOVABLE has zero managed pages (this trips people up constantly; the arithmetic is worked out on a real box in Memory Reclaim Overview). may_writepage = !laptop_mode is the one field an earlier revision of this note got wrong: with vm.laptop_mode non-zero the very first reclaim passes are forbidden from issuing writeback at all, precisely so a spun-down disk is not woken; the engine relaxes this itself once sc->priority < DEF_PRIORITY - 2.

The engine then loops, decreasing sc.priority from 12 toward 0, calling shrink_zones() → shrink_node() → shrink_lruvec() on each iteration until it has freed nr_to_reclaim folios or exhausted its passes. “Priority” is an inverted name: it is a shift count, and the comment at include/linux/mmzone.h line 1203 spells out the meaning — “A value of 12 for DEF_PRIORITY implies that we will scan 1/4096th of the queues (queue_length >> 12) during an aging round.”

sc.priority	Fraction of each LRU scanned per pass	Reading
12 (DEF_PRIORITY)	1 / 4096	First, cheapest pass — skim the tail
10	1 / 1024
9	1 / 512	< DEF_PRIORITY - 2: may_writepage is forced to 1
6	1 / 64
3	1 / 8
1	1 / 2	Last level kswapd reaches; VMSCAN_THROTTLE_NOPROGRESS can fire here
0	1 / 1 — the entire LRU	Direct reclaim only; the final, most desperate pass

The priority ladder, derived from DEF_PRIORITY = 12 and the >> priority scan-size computation in shrink_lruvec(). What it shows: the cost of a reclaim pass grows geometrically as priority falls, doubling with each step, and the last pass a direct reclaimer makes scans 4,096 times more folios than its first. The insight to take: this is why direct-reclaim stall durations have such a long tail. A stall that resolves at priority 12 is invisible; one that grinds down to priority 0 on a 100 GiB LRU has walked tens of millions of folios on the application’s own thread. The distribution of stall latencies is not roughly normal — it is a sum over a geometric series, and the tail is where your p99.9 lives.

The page-scanning, LRU-aging, dirty-writeback, and swap logic inside shrink_node() is the same machinery the background daemon uses — it is documented in Memory Reclaim Overview, The LRU Lists, and Shrinkers and Slab Reclaim, and is deliberately not re-explained here. What is specific to direct reclaim is who runs it (the allocating task), how many times it restarts, and the accounting that results.

The four-way retry structure — and why allocstall over-counts

do_try_to_free_pages() does not run its priority loop once. The retry: label sits at the very top of the function, and there are three distinct goto retry statements below the loop, each of which resets sc->priority = initial_priority and starts the whole descent over with one policy relaxed. Only after all three have been tried does the function return 0 and let the slowpath consider the OOM killer.

stateDiagram-v2
    [*] --> Descend
    Descend: Priority descent<br/>12 down to 0<br/>shrink_zones() each pass
    Descend --> Success: nr_reclaimed > 0
    Descend --> CompactReady: sc-&gt;compaction_ready<br/>(costly order, zone has<br/>enough free for compaction)
    Descend --> R1: nr_reclaimed == 0

    R1: Relax 1 — memcg_full_walk = 1<br/>stop doing partial cgroup-tree walks
    R2: Relax 2 — force_deactivate = 1<br/>deactivate even where the<br/>inactive:active estimate said no
    R3: Relax 3 — memcg_low_reclaim = 1<br/>dip into memory.low protection

    R1 --> Descend: goto retry<br/>(ALLOCSTALL counted AGAIN)
    R1 --> R2: already set
    R2 --> Descend: goto retry<br/>(ALLOCSTALL counted AGAIN)
    R2 --> R3: nothing was skipped
    R3 --> Descend: goto retry<br/>(ALLOCSTALL counted AGAIN)
    R3 --> Fail: no reserves left untapped

    CompactReady --> ReturnOne: return 1<br/>"aborted to try compaction, do not OOM"
    Success --> [*]
    ReturnOne --> [*]
    Fail: return 0 — the allocator may now consider OOM
    Fail --> [*]

The retry state machine of do_try_to_free_pages(), mm/vmscan.c at v6.12. What it shows: a single call into direct reclaim can run the full priority descent up to four times, each time with one more protection disabled — first cgroup-iteration fairness, then the inactive:active heuristic, then memory.low protection. The order is deliberate: the cheapest relaxation is tried first and the one that violates a user-declared guarantee (memory.low) is tried last. The insight to take: because __count_zid_vm_events(ALLOCSTALL, ...) sits below the retry: label, allocstall counts restarts, not entries. A machine under severe pressure can show four allocstall increments for one malloc(), so allocstall divided by uptime is an upper bound on the stall rate, not the stall rate — and the more severe the pressure, the more it over-reports. Use the mm_vmscan_direct_reclaim_begin tracepoint, which fires exactly once per entry, when you need the true count.

The sc->compaction_ready exit is the other subtlety, and it is the reason a costly high-order allocation can “stall” while barely reclaiming anything. Inside shrink_zones(), for sc->order > PAGE_ALLOC_COSTLY_ORDER (i.e. order 4 and up), the loop checks compaction_ready(zone, sc) — which compares free pages against high_wmark_pages(zone) + compact_gap(sc->order), where compact_gap(order) is 2 << order pages (include/linux/compaction.h, doubled because “compaction free scanner may have up to 1 << order pages on its list and then try to split an (order - 1) free page”). If the zone already has that much free, reclaim skips it entirely and sets compaction_ready, and do_try_to_free_pages() returns 1 — a deliberate lie that means “do not OOM-kill; the problem is fragmentation, not shortage.” See Memory Compaction.

The allocstall / PGSCAN_DIRECT counters

The very first thing do_try_to_free_pages() does, for non-cgroup reclaim, is:

if (!cgroup_reclaim(sc))
    __count_zid_vm_events(ALLOCSTALL, sc->reclaim_idx, 1);

So every entry into direct reclaim bumps allocstall (exposed per-zone-class in /proc/vmstat as allocstall_dma, allocstall_normal, allocstall_movable, etc.). This is the headline metric: allocstall counting up means tasks are being forced into synchronous reclaim. Separately, the page scanner attributes the pages it scans and steals to either the kswapd or the direct bucket via reclaimer_offset():

item = PGSCAN_KSWAPD + reclaimer_offset();   /* PGSCAN_DIRECT when not kswapd/khugepaged */
...
item = PGSTEAL_KSWAPD + reclaimer_offset();

reclaimer_offset() returns 0 when current_is_kswapd(), the khugepaged offset when current_is_khugepaged(), and otherwise the offset to the PGSCAN_DIRECT/PGSTEAL_DIRECT slot. The function is guarded by a wall of BUILD_BUG_ON()s asserting that PGSTEAL_DIRECT - PGSTEAL_KSWAPD == PGSCAN_DIRECT - PGSCAN_KSWAPD and the same for the demote and khugepaged variants — because the whole trick is that four different counter families are laid out in enum vm_event_item with identical stride, so one computed offset indexes all of them. Hence /proc/vmstat exposes pgscan_direct and pgsteal_direct (folios scanned and freed by direct reclaimers) alongside pgscan_kswapd/pgsteal_kswapd. A healthy system does most of its scanning under pgscan_kswapd; a rising pgscan_direct and allocstall together say “kswapd is not keeping up and applications are paying for it.” (See kswapd and Background Reclaim for the kswapd-side counters.)

Here is the full set of counters that move when a task stalls, with the exact increment site, because reading a counter without knowing where it is bumped is how people talk themselves into wrong conclusions:

Counter (/proc/vmstat)	Increment site at v6.12	Counts	Caveat that bites
allocstall_<zone>	do_try_to_free_pages(), below the retry: label	Entries and restarts into global direct reclaim	Over-counts by up to 4× per real entry; cgroup_reclaim(sc) is excluded entirely
pgscan_direct	shrink_inactive_list() and the MGLRU evict_folios() path, via reclaimer_offset()	Folios isolated for examination by a non-kswapd, non-khugepaged reclaimer	Includes folios that turn out to be unfreeable
pgsteal_direct	Same functions, PGSTEAL_KSWAPD + reclaimer_offset()	Folios actually freed	pgsteal_direct / pgscan_direct is the efficiency ratio worth alerting on
pgscan_direct_throttle	throttle_direct_reclaim(), immediately before the wait	Times a direct reclaimer was parked on pfmemalloc_wait instead of being allowed to scan	Non-zero means the pfmemalloc reserve, not general slowness
pgdemote_direct	Demotion path, same reclaimer_offset() stride	Folios demoted to a slower memory tier rather than freed	Zero on a machine without tiering configured
/proc/pressure/memory some/full total	psi_memstall_enter/leave in __alloc_pages_direct_reclaim() (and in balance_pgdat())	Microseconds of stall, cumulative since boot	Both reclaimers contribute; PSI alone cannot tell you which
nr_throttled_written (node stat)	__acct_reclaim_writeback()	Folios written back while somebody is writeback-throttled	The wake-up trigger for VMSCAN_THROTTLE_WRITEBACK sleepers

Where each direct-reclaim counter is incremented, mm/vmscan.c and mm/page_alloc.c at v6.12. What it shows: the counters measure four different things — entries, work, yield, and time — and only the last of them (PSI) is denominated in the unit an SLO cares about. The insight to take: allocstall answers “did it happen?”, pgscan_direct/pgsteal_direct answers “was it worth it?”, and PSI answers “did it hurt?” You need all three, and they routinely disagree: a machine can show a million allocstall events and 0.005% PSI, which is a healthy page-cache-heavy workload, not an incident.

Backoff and throttling — reclaim_throttle, not congestion_wait

A common misconception, fossilized in older documentation and blog posts, is that direct reclaimers “wait on congestion” via congestion_wait() / wait_iff_congested(). Those functions were removed. In 6.12/6.18 the backoff primitive is reclaim_throttle(pgdat, reason) (mm/vmscan.c), which puts the reclaiming task to sleep on one of four per-node wait queues keyed by an explicit reason (enum vmscan_throttle_state), with reason-specific timeouts:

Reason	Timeout	Raised where	Woken early by
VMSCAN_THROTTLE_WRITEBACK	HZ/10 ≈ 100 ms	shrink_inactive_list() when every isolated folio was unqueued-dirty on a cgroup-v1 hierarchy; and in shrink_node() when sc->nr.immediate is non-zero — but that second site is inside if (current_is_kswapd())	__acct_reclaim_writeback(), once nr_written > SWAP_CLUSTER_MAX * nr_throttled folios have completed writeback
VMSCAN_THROTTLE_ISOLATED	HZ/50 = 20 ms	shrink_inactive_list() when too_many_isolated(); also isolate_migratepages_block() in mm/compaction.c	too_many_isolated() calling wake_throttle_isolated() when the condition clears
VMSCAN_THROTTLE_NOPROGRESS	1 jiffy	consider_reclaim_throttle(), only at sc->priority == 1 with sc->nr_reclaimed == 0	Any reclaimer reaching >12.5% efficiency wakes the whole queue
VMSCAN_THROTTLE_CONGESTED	1 jiffy	End of shrink_node(), gated on !current_is_kswapd() and the node/cgroup congested bits	Timeout only

The four throttle reasons, their timeouts, and their wake conditions, all read from reclaim_throttle() and its callers in mm/vmscan.c at v6.12. What it shows: two of the four reasons have an explicit early-wake path (writeback completion and isolation clearing) and two rely on the timeout. The insight to take: the two long timeouts are the ones with early wakes, and the two that can only time out are deliberately set to a single jiffy. That is not an accident — it is the fix for a real regression, described below.

The code comment above the switch is refreshingly honest: “These figures are pulled out of thin air.” Its author says the same thing in the commit message that introduced the centralised timeouts — “The original timeout values to congestion_wait() were probably pulled out of thin air or copy&pasted from somewhere else. This patch centralises the timeout values and selects a timeout based on the reason for reclaim throttling. These figures are also pulled out of the same thin air but better values may be derived” (Mel Gorman, commit c3f4a9a2b082, merged for 5.16). He also tells you how to derive better ones: “Running a workload that is throttling for inappropriate periods and tracing mm_vmscan_throttled can be used to pick a more appropriate value. Excessive throttling would pick a lower timeout whereas excessive CPU usage in reclaim context would select a larger timeout.”

The motivation for replacing congestion_wait() is stated in the series’ cover letter: “congestion_wait has been broken for a long time … Even if congestion throttling worked, it was never a great idea. While excessive dirty/writeback pages at the tail of the LRU is one possibility that reclaim may be slow, there is also the problem of too many pages being isolated and reclaim failing for other reasons (elevated references, too many pages isolated, excessive LRU contention etc)” (commit 8cd7c588decf). The old primitive keyed off block-device congestion, a signal that stopped meaning anything on fast SSDs and stacked or virtual devices; the replacement asks why reclaim is slow and waits on the specific thing that would unblock it.

reclaim_throttle() deliberately exempts kthreads and user-workers other than kswapd:

if (!current_is_kswapd() &&
    current->flags & (PF_USER_WORKER|PF_KTHREAD)) {
	cond_resched();
	return;
}

The reason is deadlock avoidance — those threads “may be required for reclaim to make forward progress (e.g. journalling workqueues or kthreads).” Throttling the journal thread that must complete a transaction before dirty pages can be cleaned would stop the very writeback the throttle is waiting for. There is a second, older exemption of the same shape: current_may_throttle() returns false for PF_LOCAL_THROTTLE tasks, which is how loop-back nfsd threads servicing a local mount avoid throttling themselves into a hang.

The NOPROGRESS throttle, and the regression that shaped it

consider_reclaim_throttle() is called once per shrink_zones() pass, on the first pgdat in the zonelist only, and it is worth reading in full because its guard conditions are the scar tissue from a well-documented incident:

/* If reclaim is making progress greater than 12% efficiency then
 * wake all the NOPROGRESS throttled tasks. */
if (sc->nr_reclaimed > (sc->nr_scanned >> 3)) { ... wake_up(wqh); return; }
 
if (current_is_kswapd() || cgroup_reclaim(sc))
	return;
 
/* Throttle if making no progress at high prioities. */
if (sc->priority == 1 && !sc->nr_reclaimed)
	reclaim_throttle(pgdat, VMSCAN_THROTTLE_NOPROGRESS);

sc->nr_scanned >> 3 is one eighth, so “greater than 12% efficiency” is the source comment’s rounding of more than 12.5% of scanned folios freed. Any reclaimer that clears that bar wakes every task sleeping on the node’s NOPROGRESS queue — the throttle is a shared-fate mechanism, not a per-task backoff: one task making progress is taken as evidence that everyone should try again.

The three guards below it were added in December 2021 after Mike Galbraith, Alexey Avramov, and Darrick Wong independently reported multi-minute stalls. Mel Gorman’s fix commit describes the failure and the remedy precisely: “In Alexey’s case, a memory hog that should go OOM quickly stalls for several minutes before stalling … Systems at or near an OOM state that cannot be recovered must reach OOM quickly … To address this, only stall for the first zone in the zonelist, reduce the timeout to 1 tick for VMSCAN_THROTTLE_NOPROGRESS and only stall if the scan control nr_reclaimed is 0, kswapd is still active and there were excessive pages pending for writeback” (commit 1b4e3f26f9f7). The reproducer is three lines:

for i in {1..3}; do tail /dev/zero; done   # three processes each mapping all of RAM

On 5.16-rc1 this hung the machine for minutes; with the fix it completes in seconds, as it had on 5.15. Avramov’s second case — the same test while watching a video — went from “lots of frames missing and numerous audio glitches” back to smooth playback. This is the single best illustration of why reclaim throttling is dangerous rather than merely conservative: a throttle that fires when the system is genuinely unrecoverable does not save the system, it postpones the OOM kill that would have.

“kswapd is still active” is enforced by skip_throttle_noprogress(), which returns true — meaning do not throttle — in two cases:

if (pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES)   /* 16 */
	return true;                 /* kswapd has given up; near OOM, go fast */
...
if (2 * write_pending <= reclaimable)
	return true;                 /* not actually a writeback problem */

The first is the escape hatch: once the node’s kswapd has failed 16 consecutive balancing passes, the node is declared hopeless and nobody throttles on it any more, so the OOM killer is reached quickly instead of slowly. The second says there is no point sleeping for writeback if fewer than half the reclaimable folios are dirty or under writeback — the slowness must be something else, and sleeping would not help.

The pfmemalloc throttle — a different thing, do not confuse it

try_to_free_pages() opens with:

if (throttle_direct_reclaim(sc.gfp_mask, zonelist, nodemask))
    return 1;

This is not the general “reclaim is slow” backoff. throttle_direct_reclaim() is specifically the pfmemalloc-reserve backpressure for the swap-over-network / swap-over-NBD case. The source comment states the scenario: “Throttle direct reclaimers if backing storage is backed by the network and the PFMEMALLOC reserve for the preferred node is getting dangerously depleted. kswapd will continue to make progress and wake the processes when the low watermark is reached.” The deadlock it prevents is circular: swapping to an NFS or NBD device requires allocating network buffers, and allocating requires memory, which requires swapping.

allow_direct_reclaim(pgdat) computes the reserve and the verdict:

if (pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES)
	return true;                                   /* hopeless node: never throttle */
for (i = 0; i <= ZONE_NORMAL; i++) {
	if (!managed_zone(zone) || !zone_reclaimable_pages(zone))
		continue;                              /* zones with nothing to reclaim don't count */
	pfmemalloc_reserve += min_wmark_pages(zone);
	free_pages          += zone_page_state_snapshot(zone, NR_FREE_PAGES);
}
if (!pfmemalloc_reserve)
	return true;                                   /* unexpected config: do not throttle */
wmark_ok = free_pages > pfmemalloc_reserve / 2;

Three things in there are easy to miss. The reserve is summed only over ZONE_NORMAL and below, because a network buffer allocated with GFP_KERNEL cannot come from highmem. Zones with zone_reclaimable_pages() == 0 are skipped on both sides of the comparison — a 2017 fix from Johannes Weiner (commit d450abd81b08, “mm: fix check for reclaimable pages in PF_MEMALLOC reclaim throttling”) — so a zone full of pinned memory neither inflates the reserve nor contributes free pages. And the function has a side effect: if the watermark is not ok, it clamps pgdat->kswapd_highest_zoneidx down to ZONE_NORMAL and wakes kswapd itself, on the reasoning in the source that “kswapd must be awake if processes are being throttled.”

The wait itself is where an earlier revision of this note — and, at the time of writing, the parent Memory Reclaim Overview — glossed over a genuinely important branch. There are two waits, not one, and only one of them has a timeout:

if (!(gfp_mask & __GFP_FS))
	wait_event_interruptible_timeout(pgdat->pfmemalloc_wait,
		allow_direct_reclaim(pgdat), HZ);          /* bounded: 1 second */
else
	wait_event_killable(zone->zone_pgdat->pfmemalloc_wait,
		allow_direct_reclaim(pgdat));              /* UNBOUNDED */

A caller that cannot enter the filesystem (GFP_NOFS, GFP_NOIO) gets a one-second bounded wait, and the comment explains why: such a caller may be holding an FS lock or be mid-journal-transaction, and “it is not safe to block on pfmemalloc_wait as kswapd could be blocked waiting on the same lock.” A normal GFP_KERNEL caller gets wait_event_killable() with no timeout at all — it sleeps until kswapd calls wake_up_all(&pgdat->pfmemalloc_wait) from balance_pgdat() or prepare_kswapd_sleep(), or until somebody sends it a fatal signal. That is the single longest thing a direct reclaimer can do, and it is invisible in pgscan_direct because the task never got to scan anything. It is visible in exactly two places: pgscan_direct_throttle, and PSI.

Each such throttle bumps PGSCAN_DIRECT_THROTTLE (pgscan_direct_throttle in /proc/vmstat). Kernel threads (PF_KTHREAD) and tasks with a fatal signal pending are exempted before the check even runs — they may be needed to clean pages, or are exiting and about to free memory. The returned 1 (rather than 0) on a fatal-signal abort tells the allocator not to OOM-kill at this point, since a task is already dying.

Keep the two throttles distinct — they are constantly conflated:

	throttle_direct_reclaim()	reclaim_throttle()
Where	Very first line of try_to_free_pages(), before any scanning	Inside shrink_inactive_list() / shrink_node() / consider_reclaim_throttle(), during scanning
Guards	The pfmemalloc reserve (½ the summed min watermark below ZONE_NORMAL)	Four distinct slowness causes, one wait queue each
Wait queue	pgdat->pfmemalloc_wait	pgdat->reclaim_wait[NR_VMSCAN_THROTTLE] (4 queues)
Duration	Unbounded (__GFP_FS) or 1 s (!__GFP_FS)	1 jiffy, 20 ms, or 100 ms by reason
Applies to kswapd	No	Only VMSCAN_THROTTLE_WRITEBACK, and only via the nr.immediate site
Counter	pgscan_direct_throttle	none — visible only via the mm_vmscan_throttled tracepoint
Purpose	Deadlock avoidance for swap-over-network	Reduce futile scanning and LRU contention

The two throttles side by side. What it shows: they sit at different points in the call stack, wait on different queues, have wildly different durations, and only one of them is counted in /proc/vmstat. The insight to take: if pgscan_direct_throttle is non-zero you have a reserve problem and should look at min_free_kbytes and network-backed swap; if it is zero but PSI full is high, your stalls are inside reclaim_throttle() and are invisible to vmstat — you need mm_vmscan_throttled to see them at all.

The Anatomy of One Stall

Everything above is static structure. Here is the same machinery as a timeline — one ordinary task, one GFP_KERNEL order-0 allocation, on a node where kswapd is already awake and losing.

sequenceDiagram
    autonumber
    participant App as Task (your service thread)
    participant Alloc as __alloc_pages_slowpath
    participant PSI as PSI accounting
    participant TTFP as try_to_free_pages
    participant WQ as pgdat wait queues
    participant KS as kswapd<N>
    participant IO as writeback / swap I/O

    App->>Alloc: page fault -> alloc_pages(GFP_KERNEL, 0)
    Alloc->>KS: wake_all_kswapds() (already awake — no-op)
    Alloc->>Alloc: get_page_from_freelist(ALLOC_WMARK_MIN)
    Note over Alloc: fails — free < min in every eligible zone
    Alloc->>PSI: psi_memstall_enter() — STALL CLOCK STARTS
    Alloc->>TTFP: __perform_reclaim() -> try_to_free_pages()

    TTFP->>TTFP: throttle_direct_reclaim(): allow_direct_reclaim()?
    alt pfmemalloc reserve below half
        TTFP->>WQ: sleep on pfmemalloc_wait (UNBOUNDED for __GFP_FS)
        KS->>WQ: wake_up_all(&pfmemalloc_wait) after a balancing pass
        WQ-->>TTFP: woken; return 1 without scanning
    end

    loop sc.priority 12 down to 0
        TTFP->>TTFP: shrink_zones() -> shrink_node() -> shrink_lruvec()
        TTFP->>IO: writeback dirty file folios / swap out anon folios
        opt too_many_isolated() — other reclaimers hold the LRU
            TTFP->>WQ: reclaim_throttle(ISOLATED) — 20 ms
        end
        opt node/lruvec marked congested BY KSWAPD
            TTFP->>WQ: reclaim_throttle(CONGESTED) — 1 jiffy
        end
        opt priority == 1 and nothing reclaimed
            TTFP->>WQ: reclaim_throttle(NOPROGRESS) — 1 jiffy
        end
    end

    TTFP-->>Alloc: nr_reclaimed (target was just 32 folios)
    Alloc->>Alloc: get_page_from_freelist() again
    opt still no page
        Alloc->>Alloc: unreserve_highatomic_pageblock() + drain_all_pages() (IPI all CPUs)
        Alloc->>Alloc: get_page_from_freelist() once more
    end
    Alloc->>PSI: psi_memstall_leave() — STALL CLOCK STOPS
    Alloc-->>App: page (finally) — this interval is your latency spike

One direct-reclaim episode end to end, composed from __alloc_pages_slowpath() / __alloc_pages_direct_reclaim() in mm/page_alloc.c and try_to_free_pages() / shrink_node() in mm/vmscan.c at v6.12. What it shows: there are five distinct places the task can sleep inside a single allocation — the pfmemalloc wait, three reclaim_throttle() reasons, and the block-layer waits inside writeback — and all five are inside the PSI bracket. The insight to take: the congested flags the task obeys at step “node/lruvec marked congested” were set by kswapd, in shrink_node(), under if (current_is_kswapd()). The daemon diagnoses the congestion; the application serves the sentence. That single asymmetry is the cleanest statement of the relationship between the two reclaimers: kswapd decides how bad things are, and direct reclaimers are the ones who wait for it to get better.

Why Direct Reclaim Is a Sign of Pressure

Walk the slow path again with the timing in mind. The fast path failed (free < low watermark in every eligible zone). kswapd was woken. The min-watermark retry failed. Then the task reclaimed itself. For all of that to happen, the per-node kswapd — which is supposed to keep free memory between the low and high watermarks — was either not yet awake, or awake but unable to free pages as fast as allocations were consuming them. Direct reclaim is the system saying: background reclaim could not stay ahead of demand, so I am charging the cost to the application. That is qualitatively different from kswapd running — kswapd running is normal and asynchronous; direct reclaim running means a thread is blocked.

The cost is real and measurable. The blocked task accrues PSI memory-stall time; it may sleep in reclaim_throttle() for tens to hundreds of milliseconds; and if many threads hit the slow path at once they serialize on shared reclaim state and on the dirty-page writeback they are all waiting for. For latency-sensitive services this manifests as tail-latency spikes that correlate with allocstall/pgscan_direct climbing in /proc/vmstat.

Configuration / Diagnosis

There is no knob to “turn off” direct reclaim (short of using non-blocking GFP flags in your own kernel code). The lever is to make kswapd start earlier and reclaim more, so the fast path rarely fails. The two relevant sysctls (kernel vm docs):

# Widen the gap between the watermarks so kswapd wakes sooner and
# reclaims more before sleeping. Units: fraction of 10000 of node memory.
# Default 10 (= 0.1%). Raising it pushes the low/high lines further apart.
sysctl vm.watermark_scale_factor=200      # 2% of node memory
 
# Raise the absolute reserve floor (the min watermark). Bigger min ->
# bigger low/high -> kswapd has more headroom before tasks stall.
sysctl vm.min_free_kbytes=262144          # 256 MiB

The watermark arithmetic (from mm/page_alloc.c:__setup_per_zone_wmarks) makes the relationship concrete. With gap = max(min/4, managed_pages * watermark_scale_factor / 10000):

zone->_watermark[WMARK_LOW]  = min_wmark_pages(zone) + gap;
zone->_watermark[WMARK_HIGH] = low_wmark_pages(zone) + gap;

So low and high are equally spaced above min, and watermark_scale_factor controls that spacing. A larger spacing means kswapd is woken further from exhaustion (more slack before the fast path fails) and reclaims further past the danger line before sleeping — directly reducing the probability of direct reclaim. See Watermarks and the Allocation Fast Path for the watermark model and kswapd and Background Reclaim for how kswapd consumes these lines.

Diagnosis recipe:

# Direct-reclaim activity: these climbing = applications stalling on memory.
grep -E 'allocstall|pgscan_direct|pgsteal_direct|pgscan_kswapd|pgscan_direct_throttle' /proc/vmstat
 
# How much wall-clock time tasks lost stalled on memory (PSI):
cat /proc/pressure/memory      # 'some'/'full' avg10/avg60/avg300 + total (us)
 
# Trace each direct-reclaim episode (begin/end, nr_reclaimed):
echo 1 > /sys/kernel/tracing/events/vmscan/mm_vmscan_direct_reclaim_begin/enable
echo 1 > /sys/kernel/tracing/events/vmscan/mm_vmscan_direct_reclaim_end/enable
cat /sys/kernel/tracing/trace_pipe

The mm_vmscan_direct_reclaim_begin/_end tracepoints are emitted by try_to_free_pages() itself and give per-episode latency and pages freed — the most direct view of who is stalling and for how long.

Recognising a Direct-Reclaim Stall in Production

This is the section to read when a service’s p99 has fallen over and somebody has said the word “memory.” The difficulty is that direct reclaim produces no log line, no error, and no failed syscall. It produces slowness, distributed across whichever threads happened to allocate during the window, and the machine looks healthy afterwards. Everything below is about establishing, from evidence rather than vibes, whether direct reclaim is what happened.

The signature, and its three impostors

Direct reclaim has a recognisable fingerprint, and three common conditions that look like it and are not:

Observation	Direct reclaim	Impostor 1: dirty-page writeback throttling	Impostor 2: cgroup memory.high throttling	Impostor 3: THP / compaction stalls
Stack of the stalled thread	try_to_free_pages → do_try_to_free_pages → shrink_node	balance_dirty_pages	mem_cgroup_handle_over_high → try_to_free_mem_cgroup_pages	__alloc_pages_direct_compact → compact_zone
allocstall_* moving	Yes	No	No — cgroup_reclaim(sc) excludes it	Only if reclaim also ran
pgscan_direct moving	Yes	No	No (cgroup scans land in memcg stats, not global pgscan_direct)	Maybe
/proc/pressure/memory	some and often full rise	io pressure rises, not memory	memory.pressure inside the cgroup rises; host-level may not	memory rises
/proc/pressure/io	May rise as a consequence	Primary signal	Depends	Usually flat
Correlates with	Free memory near WMARK_MIN	dirty_ratio being hit	The cgroup’s own usage crossing memory.high	High-order allocations, nr_anon_transparent_hugepages
The fix that works	Widen the watermark band, add memory, reduce footprint	vm.dirty_background_bytes, faster storage	Raise memory.high or shrink the workload	THP=madvise, or defrag=defer

Direct reclaim versus the three conditions most often mistaken for it. What it shows: four different stalls that all present as “the app got slow and the machine was using a lot of memory”, separated by which counter moves. The insight to take: the discriminating test is cheap and takes one command — allocstall and pgscan_direct move for global direct reclaim and for nothing else. If they are flat across the incident window, whatever happened was not this, and the two pressure files will tell you which of the impostors it was instead. Notably, per-cgroup reclaim under memory.high is explicitly excluded from allocstall by the if (!cgroup_reclaim(sc)) guard, which is why a containerised service can stall on memory all day with a completely flat host-level allocstall; see Per-cgroup Reclaim and Memory Pressure.

The triage sequence

flowchart TD
  START["Latency incident.<br/>Suspect memory."]
  PSI{"/proc/pressure/memory<br/>'some avg60' &gt; 0 during<br/>the incident window?"}
  NOTMEM["Not a memory stall.<br/>Check /proc/pressure/io and<br/>/proc/pressure/cpu instead."]
  AS{"Did allocstall_* and<br/>pgscan_direct move<br/>across the window?"}
  CG{"Is the workload in a cgroup<br/>with memory.high set<br/>below memory.max?"}
  CGR["Per-cgroup reclaim.<br/>Read memory.pressure and<br/>memory.stat inside the cgroup.<br/>allocstall is blind to this."]
  THR{"pgscan_direct_throttle<br/>non-zero?"}
  RESERVE["pfmemalloc reserve exhaustion.<br/>Look at network-backed swap,<br/>raise vm.min_free_kbytes."]
  EFF{"pgsteal_direct / pgscan_direct<br/>&lt; ~30% ?"}
  LATE["Reclaim is running late and<br/>scanning junk. kswapd lost the race.<br/>-&gt; widen the band:<br/>vm.watermark_scale_factor"]
  ORD{"Are the stalls on high-order<br/>allocations? (THP faults,<br/>order&gt;3 in trace)"}
  COMPACT["Fragmentation, not shortage.<br/>See Memory Compaction.<br/>Reclaim is a symptom here."]
  CAP["Genuine capacity problem.<br/>Reclaim works but there is<br/>simply not enough memory.<br/>-&gt; add RAM or cut footprint."]
  FULL{"PSI 'full' &gt;&gt; 0<br/>relative to 'some'?"}
  WB["Everything is blocked at once:<br/>look for writeback backlog<br/>(nr_dirty, nr_writeback) and<br/>slow backing devices."]

  START --> PSI
  PSI -->|"no"| NOTMEM
  PSI -->|"yes"| AS
  AS -->|"no"| CG
  CG -->|"yes"| CGR
  CG -->|"no"| NOTMEM
  AS -->|"yes"| THR
  THR -->|"yes"| RESERVE
  THR -->|"no"| FULL
  FULL -->|"yes"| WB
  FULL -->|"no"| EFF
  EFF -->|"yes"| LATE
  EFF -->|"no"| ORD
  ORD -->|"yes"| COMPACT
  ORD -->|"no"| CAP

A triage decision tree for a suspected direct-reclaim stall, built from the counters and their increment sites established above. What it shows: the order matters — PSI first (did anything actually lose time?), then allocstall (was it global direct reclaim at all?), then the cheap discriminators, and only at the end the expensive question of whether the machine is simply out of memory. The insight to take: three of the seven leaves are not “add memory”. The most common real outcome on a large-RAM box is the LATE leaf — reclaim works fine, it just started too late, and the fix is vm.watermark_scale_factor, not hardware.

Getting per-stall latency, not just counters

Counters tell you a stall happened; they do not tell you it was 3 ms or 300 ms, and the difference decides whether you care. Two routes, in increasing order of precision.

Route 1 — PSI, cheap and always on. /proc/pressure/memory’s total field is cumulative microseconds, so sampling it twice gives you real lost time over a real window with no tracing overhead:

# Wall-clock memory-stall time over 60 seconds, as a percentage.
a=$(awk '/^some/{print $NF}' /proc/pressure/memory | tr -d 'total=')
sleep 60
b=$(awk '/^some/{print $NF}' /proc/pressure/memory | tr -d 'total=')
echo "some: $(( (b-a) / 600000 ))% of wall clock"   # (Δµs / 60e6) * 100

The some/full split is the part worth internalising. Per psi.rst, some is time during which at least one runnable task was stalled on memory; full is time during which no task could make progress. full approaching some means your stalls are system-wide and simultaneous — the shape of a writeback backlog or a synchronised allocation burst — while full near zero with a high some means a few unlucky threads are absorbing the pain while the machine as a whole keeps working. The former is an incident; the latter is often just a busy page cache. Full treatment in Pressure Stall Information.

Route 2 — the tracepoints, exact and per-episode. try_to_free_pages() brackets do_try_to_free_pages() with trace_mm_vmscan_direct_reclaim_begin(order, sc.gfp_mask) and trace_mm_vmscan_direct_reclaim_end(nr_reclaimed). These fire once per entry, unlike allocstall, so they are the ground truth for stall count as well as duration:

# Per-episode direct-reclaim latency histogram, and who is paying.
cd /sys/kernel/tracing
echo 1 > events/vmscan/mm_vmscan_direct_reclaim_begin/enable
echo 1 > events/vmscan/mm_vmscan_direct_reclaim_end/enable
echo 1 > events/vmscan/mm_vmscan_throttled/enable      # the sleeps inside the stall
cat trace_pipe

mm_vmscan_throttled is the one people forget, and it is the only visibility into reclaim_throttle() at all. Its fields are nid, usec_timeout, usec_delayed, and a decoded reason — so it tells you not just that a task slept but which of the four reasons applied and whether it slept the full timeout or was woken early. A trace full of reason=VMSCAN_THROTTLE_WRITEBACK usec_delayed=100000 is a storage problem wearing a memory costume.

For a histogram rather than a firehose, the same two tracepoints under bpftrace give a distribution in a few lines:

bpftrace -e '
tracepoint:vmscan:mm_vmscan_direct_reclaim_begin { @s[tid] = nsecs; }
tracepoint:vmscan:mm_vmscan_direct_reclaim_end   /@s[tid]/ {
    @us = hist((nsecs - @s[tid]) / 1000);
    @by_comm[comm] = sum((nsecs - @s[tid]) / 1000);
    delete(@s[tid]);
}'

@by_comm is the number that ends arguments: it attributes total stalled microseconds to specific processes, which distinguishes “our service is the victim of a noisy neighbour” from “our service is the noisy neighbour.”

Uncertain

Verify: that @by_comm attribution is a fair statement of blame. Reason: a thread stalls because the node is short of memory, and the thread that stalls is simply the next one to allocate — which correlates with allocation rate, not with the footprint that caused the shortage. A thread that allocated nothing and touched no memory can be the one that pays. To resolve: pair the stall attribution with per-cgroup memory.current and memory.events deltas over the same window, which do measure footprint. uncertain

What to change, in order of how often it is the right answer

1. Widen the low..min band — vm.watermark_scale_factor. This is the correct fix for the most common real case: reclaim is capable but starts too late for the workload’s burst rate. Raising it from the default 10 (0.1%) to 100–500 (1–5%) gives kswapd proportionally more runway. The kernel’s own documentation names this exact symptom: “A high rate of threads entering direct reclaim (allocstall) or kswapd going to sleep prematurely (kswapd_low_wmark_hit_quickly) can indicate that the number of free pages kswapd maintains for latency reasons is too small for the allocation bursts occurring in the system” (vm.rst). The mechanics of what this does to the daemon are in kswapd and Background Reclaim.
2. Raise the floor — vm.min_free_kbytes. Scales min, and therefore low and high with it. Prefer watermark_scale_factor unless pgscan_direct_throttle is non-zero, in which case it is the reserve itself that is too small and this is the knob that grows it.
3. Give the workload a proactive-reclaim budget — in a container, set memory.high below memory.max so the cgroup reclaims itself gradually instead of hitting a wall. Or drive memory.reclaim from userspace on a schedule. Both are covered in Per-cgroup Reclaim and Memory Pressure.
4. Reduce dirty-page backlog — if PSI full dominates and mm_vmscan_throttled shows WRITEBACK reasons, the reclaimer is waiting on storage. vm.dirty_background_bytes and vm.dirty_bytes (absolute, not the ratio forms) bound how much dirty data can accumulate before flushing starts.
5. Stop asking for high-order pages — if the stalls are THP faults, transparent_hugepage/defrag=defer or madvise moves the work off the fault path entirely. See Transparent Huge Pages and Memory Compaction.
6. Add memory. Last, not first, and only once the counters have ruled out the five cheaper answers.

Failure Modes and Common Misunderstandings

• “Direct reclaim means the system is out of memory.” No — it means the system is short right now relative to the watermarks, and kswapd is behind. The OOM killer is much later: only after reclaim makes zero progress across the retry loop does the slow path consider OOM (see The OOM Killer). Direct reclaim is the warning, not the funeral.
• Confusing the two throttles. reclaim_throttle() (4 reasons, general backoff inside shrink_node) is not throttle_direct_reclaim() (pfmemalloc reserve, swap-over-network). Older write-ups also still mention congestion_wait() — that primitive is gone in 6.12/6.18.
• Assuming atomic allocations stall. GFP_ATOMIC and any non-blocking context clear __GFP_DIRECT_RECLAIM; they fail fast rather than reclaim. A driver doing GFP_ATOMIC in an interrupt never enters this path — it just gets NULL (and must handle it). See GFP Flags and Allocation Contexts.
• Thinking direct reclaim only frees pages for you. try_to_free_pages() reclaims into the zone’s free lists; the pages you free may be grabbed by another concurrent allocator before your retry. The drain-and-retry-once logic in __alloc_pages_direct_reclaim() partly compensates, but under heavy concurrency a thread can reclaim and still not get a page.
• High-order amplification. Reclaiming for an order-0 page is cheap; reclaiming (and then compacting) for a high-order/THP allocation can be very expensive, which is why costly high-order requests often carry __GFP_NORETRY and bail to compaction or failure rather than grind in reclaim. See Memory Compaction and Page Order and Fragmentation.
• Reading allocstall as an entry count. It is a restart count. do_try_to_free_pages() can restart its whole priority descent three times (cgroup-walk, forced deactivation, memory.low), and the counter sits below the retry: label, so one blocked malloc() can produce four increments. Any dashboard that reports “stalls per second” from allocstall alone over-reports, and over-reports worse exactly when things are worst. The mm_vmscan_direct_reclaim_begin tracepoint is the honest count.
• Assuming the stall is proportional to the work. Five of the sleeps in a direct-reclaim episode are not scanning at all: the pfmemalloc wait (unbounded), three reclaim_throttle() reasons, and the block-layer wait inside writeback. A task can show a 200 ms stall with a handful of folios scanned. pgscan_direct is a work counter, not a time counter; only PSI measures time.
• Believing pgscan_direct == 0 means “no memory stalls”. Per-cgroup reclaim under memory.high is excluded from every global counter in this note by if (!cgroup_reclaim(sc)). On a container host, the interesting stalls may be entirely inside cgroups and completely invisible in /proc/vmstat. Read memory.pressure and memory.stat per cgroup.
• Treating a throttle as protective. It is a bet that waiting will help. When the machine is genuinely unrecoverable the bet loses, and the throttle converts a fast OOM kill into a multi-minute hang — precisely the 5.16 regression described above. That is why every throttle path now has an explicit “kswapd has given up, stop throttling” escape (kswapd_failures >= MAX_RECLAIM_RETRIES).
• Expecting the pages you freed to be yours. try_to_free_pages() frees into the buddy allocator globally. Under concurrency your 32 folios may be gone before your get_page_from_freelist() retry runs, which is what the drain_all_pages() fallback is trying to salvage — and why a thread can stall repeatedly without ever succeeding on the first retry.

Alternatives and When Each Fires

• kswapd and Background Reclaim — the asynchronous, per-node alternative. Ideally all reclaim is kswapd’s; direct reclaim is the fallback when kswapd cannot keep pace. Tuning the watermarks (above) shifts work from direct reclaim onto kswapd.
• Direct compaction — for high-order allocations, the slow path also tries __alloc_pages_direct_compact(), which migrates pages to assemble a contiguous run rather than freeing pages outright. Reclaim frees; compaction defragments.
• The OOM Killer — the terminal alternative: when reclaim makes no progress and the allocation cannot fail, kill a process.
• Failing the allocation — for __GFP_NORETRY/atomic requests, the “alternative” is simply returning NULL and letting the caller cope.

Production Notes

What changed after the 6.12 pin

Anything below is dated to the release that introduced it and is not present at the 6.12 pin. Verified by reading the corresponding tags.

Change	Landed	Effect on this note
defrag_mode sysctl; pgdat_balanced() checks NR_FREE_PAGES_BLOCKS when defrag_mode && order	6.15 (absent at v6.14, present at v6.15)	Changes when kswapd considers itself done, and hence how often direct reclaim is reached. Off by default.
pgdat->kswapd_failures becomes atomic_t	by 6.18 (int at v6.12)	Cosmetic for readers; matters if you are writing a BPF program that reads the field.
for_each_managed_zone_pgdat() iterator macro	by 6.18	Purely a refactor; allow_direct_reclaim() and pgdat_balanced() read differently but behave the same.
try_to_free_pages(), throttle_direct_reclaim(), allow_direct_reclaim(), reclaim_throttle()	—	Byte-identical at v6.12 and v6.18. Everything this note says about the stall path holds on both.

The shape of a real incident

On busy multi-tenant hosts, direct reclaim is the usual culprit behind unexplained latency cliffs: a memory-hungry neighbor pushes a node below its watermarks, kswapd falls behind, and unrelated latency-sensitive tasks get conscripted into try_to_free_pages() and lose tens of milliseconds each. The modern diagnostic chain is PSI first (/proc/pressure/memory “some” rising → tasks are stalling on memory), then /proc/vmstat allocstall/pgscan_direct to confirm it is direct reclaim, then the mm_vmscan_direct_reclaim_* tracepoints to attribute it. The standard mitigations are raising vm.watermark_scale_factor / vm.min_free_kbytes so kswapd starts earlier, and — in containerized environments — setting memory.high below memory.max so a cgroup does proactive, throttled reclaim before it ever hits the hard limit and triggers synchronous reclaim or OOM (see Per-cgroup Reclaim and Memory Pressure).

The best-documented public failure of the direct-reclaim path is not a capacity incident at all but the 5.16 throttling regression, and it is worth keeping in mind as the shape of the other way this goes wrong. Three independent reporters — Mike Galbraith, Alexey Avramov, and Darrick Wong — hit multi-minute stalls on desktops and small memcg environments after VMSCAN_THROTTLE_NOPROGRESS was introduced. The reproducer was for i in {1..3}; do tail /dev/zero; done: three processes each mapping all of memory, a workload that should reach OOM in seconds. Instead the machine sat in reclaim_throttle(), sleeping, being woken, and sleeping again, while the OOM killer that would have resolved it in one step never ran. Avramov’s video-playback test showed the user-visible version: dropped frames and audio glitches under a load that 5.15 had handled smoothly (commit 1b4e3f26f9f7). The lesson generalises well beyond reclaim: a backoff added to reduce futile work becomes a liveness bug the moment the condition it waits for cannot clear, which is why every throttle in this path now carries an explicit “if kswapd has given up, do not wait” escape.

Two operational habits follow from everything above. First, collect PSI, not just vmstat — the ratios in /proc/vmstat are cumulative since boot and cannot tell you what happened in the last five minutes, while /proc/pressure/memory gives both decayed averages and a microsecond total you can difference over any window. Second, keep the direct-reclaim tracepoints in your incident runbook rather than your always-on telemetry: they are cheap to enable for the duration of an investigation, they answer the question counters cannot (how long was any single stall?), and mm_vmscan_throttled is the only way to see the sleeps at all.

Measured evidence that the split is tunable

The clearest published numbers on shifting work from direct reclaim back onto kswapd come from Johannes Weiner’s defrag_mode series, which changed kswapd’s balance target from “one suitable zone at the high watermark in order-0 pages” to “the high watermark met entirely in whole pageblocks.” The commit reports a before/after on the same workload (commit a211c6550efc, March 2025):

Metric	DEFRAGMODE-ASYNC	With the new watermarks	Change
Alloc stall	2,424.60	638.87	−73.62%
Pages direct scanned	722,094	355,173	−50.81%
Pages direct reclaimed	107,258	31,163	−70.95%
Pages kswapd scanned	2,657,018	4,002,186	+50.63%
Pages kswapd reclaimed	559,583	718,578	+28.41%
THP fault fallback	11,581	5,412	−53.26%
Hugealloc time (mean)	34,300 µs	28,904 µs	−15.73%

Measured effect of making kswapd work harder, from the defrag_mode watermark commit. What it shows: direct-reclaim scanning halved and direct-reclaim stalls fell by nearly three quarters, while kswapd scanning rose by half — the total reclaim work went up 28.95%, and the workload still got faster. The insight to take: this is the whole thesis of the two-reclaimer design expressed as a measurement. Reclaim work is not the thing to minimise; reclaim work on the application’s thread is. Buying a 50% increase in background scanning to remove 74% of stalls is an unambiguously good trade, and it is the same trade vm.watermark_scale_factor offers on any kernel.

Two caveats before borrowing those numbers. defrag_mode is a 6.15 feature — it does not exist at the 6.12 pin (verified by existence-check: defrag_mode appears 10 times in mm/page_alloc.c at v6.15 and zero times at v6.14) — and it is off by default. And the workload was THP-heavy; a workload that never asks for high-order pages will not see the THP columns move at all. The allocstall result generalises; the rest of the table may not.

Uncertain

Verify: the exact reclaim_throttle() timeout values (HZ/10, HZ/50, 1 jiffy) and the four vmscan_throttle_state reasons. Reason: these are read directly from mm/vmscan.c at v6.12 and re-read at v6.18, where they are unchanged — but the author’s own commit message says the figures “are also pulled out of the same thin air,” and the values have already been changed once (the NOPROGRESS timeout was cut from a longer value to 1 tick in 1b4e3f26f9f7). They are heuristics, not contracts. To resolve: re-read the switch (reason) block in reclaim_throttle() at the exact kernel you are running before quoting millisecond figures in a tuning document, and use the mm_vmscan_throttled tracepoint’s usec_timeout field to observe the live value rather than trusting a written-down one. uncertain

Uncertain

Verify: that vm.watermark_scale_factor in the 100–500 range is the right starting point for large-memory hosts. Reason: the figure is widely repeated and follows directly from the arithmetic (0.1% of 256 GiB is only ~256 MiB of runway), and the kernel documentation endorses the direction, but no primary source consulted here states a recommended value for large machines — the documentation gives only the default (10) and the maximum (3000). To resolve: measure. Raise it stepwise and watch allocstall, pgscan_direct/pgscan_kswapd, and PSI some together; the correct value is workload-specific and the cost of overshooting is a permanently larger free-memory reserve, i.e. less page cache. uncertain

See Also

• kswapd and Background Reclaim — the asynchronous counterpart, and the note that owns the daemon: balance_pgdat(), the sleep/wake cycle, watermark boosting, and the kcompactd handoff. Direct reclaim is what happens when that daemon can’t keep up.
• Watermarks and the Allocation Fast Path — the min/low/high watermarks whose breach triggers the slow path.
• Memory Reclaim Overview — the umbrella: what reclaim is, the watermark arithmetic on a measured box, victim selection, MGLRU, shrinkers, and the shared shrink_node/LRU machinery both reclaimers use.
• Transparent Huge Pages · Page Order and Fragmentation — why a high-order allocation stalls differently from an order-0 one.
• The LRU Lists · Shrinkers and Slab Reclaim — what the scanner actually walks and frees.
• GFP Flags and Allocation Contexts — how __GFP_DIRECT_RECLAIM decides whether a request may stall.
• Pressure Stall Information — PSI brackets every direct-reclaim episode.
• The OOM Killer — the terminal step when reclaim makes no progress.
• Linux Memory Management MOC — parent map (§8 Memory Reclaim and the LRU).