MGLRU Generations and Aging

In the Multi-Generational LRU (MGLRU), evictable pages in each lruvec (the per-memory-cgroup, per-NUMA-node reclaim unit) are partitioned into a small sliding window of generations ordered by access recency. The youngest generation’s sequence number is held in lrugen->max_seq, the oldest in lrugen->min_seq[], and both are monotonically increasing 64-bit counters; the window holds between MIN_NR_GENS (2) and MAX_NR_GENS (4) live generations (include/linux/mmzone.h, v6.12). Aging is the producer half of reclaim: it scans for recently-accessed pages, promotes the hot ones to the youngest generation, and increments max_seq to mint a new youngest generation — implicitly demoting everything that was not promoted one step toward the oldest generation, where eviction drains them. Each generation is further subdivided into tiers by access frequency, and a refault-feedback loop decides which tiers to protect. This note dissects that machinery; for orientation read Multi-Generational LRU first, and for the scanning details read MGLRU Page Table Walks.

What this note owns

MGLRU is documented across four notes in this vault and the boundaries are deliberate. This note owns the bookkeeping: what a generation is, what a tier is, what every field of struct lru_gen_folio does, when the aging decides to mint a new generation, how the eviction decides what to sort where, and how the refault feedback loop turns those counters into a protection decision. It does not re-explain why MGLRU exists or how to switch it on — Multi-Generational LRU owns motivation, the enabled bitmask, min_ttl_ms and the release/governance timeline. It does not explain how the accessed bit is physically sampled — MGLRU Page Table Walks owns walk_page_range(), lru_gen_look_around() and the Bloom filters. It does not argue MGLRU against the old scanner — MGLRU vs the Classic LRU owns that, and The LRU Lists owns the classic active/inactive machinery being replaced.

The constants, in one place

Every number this note uses is a compile-time constant. Collecting them up front makes the rest readable, because MGLRU’s design is essentially a set of consequences of these six values. All were read from Linux v6.12 during this write-up:

ConstantValueDefined inWhat it boundsWhy this value
MIN_NR_GENS2Ummzone.h:359fewest live generations in the window“second chance” needs the accessed bit checked twice — once for the bit set by the initial fault, once to see whether the page was used since
MAX_NR_GENS4Ummzone.h:360most live generations; also the size of every per-generation array“twice the number of categories of the active/inactive LRU”; costs order_base_2(MAX_NR_GENS+1) = 3 bits in folio->flags
MAX_NR_TIERS4Ummzone.h:382access-frequency buckets inside each generation“twice the number of categories” again; uses MAX_NR_TIERS-2 = 2 spare bits in folio->flags on top of PG_referenced + PG_workingset
NR_HIST_GENS1U, or MAX_NR_GENS (= 4) with CONFIG_LRU_GEN_STATS=ymmzone.h:411-413how many generations of refault history are retainedone is enough to drive the controller; four is a debugging luxury with “a per-memcg and per-node memory overhead” (mm/Kconfig)
MIN_LRU_BATCHBITS_PER_LONG (64 on 64-bit)mmzone.h:406smallest useful batch; also the noise floor in the refault comparisonone machine word’s worth of bits, so a batch maps onto a single-word bitmap
MAX_LRU_BATCHMIN_LRU_BATCH * 64 = 4096mmzone.h:407folios moved per lru_lock acquisitioncaps lock hold time during generation rolls and state migration
MEMCG_NR_GENS3mmzone.h:542slots in the memcg LRU (only two are valid generations)a third slot so a lockless reader cannot mistake a stale seq-1 for “young”
MEMCG_NR_BINS8mmzone.h:543shards per memcg generationlets concurrent global-reclaim threads start at random bins without colliding

MGLRU’s compile-time constants. What it shows: none of these are tunable at runtime — there is no sysctl that changes the number of generations or tiers, and no Kconfig symbol either. The insight to take: the “4” in MAX_NR_GENS is not an arbitrary choice or a tuning parameter; it is the largest value that fits alongside the tier bits in the spare bits of folio->flags. Raising it means finding more page flags, which is exactly the constraint Kairui Song raised at the 2026 LSFMM+BPF planning discussion — his proposal to move the generation number out of folio->flags would “make systems with up to 63 generations possible while freeing the three page flags currently used by MGLRU” (Corbet, LWN, 2026-03-05).

Version pin

Unless stated otherwise, every code excerpt, line number and constant in this note was read from the Linux v6.12 tree (the vault’s pin — 6.12 is a longterm release, though not the only one; 6.18 is also longterm as of 2026-09-11). Checked during this write-up by fetching both files at both tags: Documentation/mm/multigen_lru.rst is byte-identical from v6.12 through v7.2, and the # multi-gen LRU { block of mm/Kconfig is byte-identical across v6.12, v6.16, v6.17, v6.18, v7.0, v7.1 and v7.2 (29 lines, same MD5). The one substantive ABI change in the window is SWAPPINESS_ANON_ONLY, absent from include/linux/swap.h at v6.15 and present at v6.16 — it widens the debugfs - verb’s swappiness argument to [0-200, max], with max meaning anonymous-only reclaim (v7.2 admin guide). The live measurements in this note come from a 7.1.13-200.fc44 kernel, which is a different tree; every section that mixes the two says which is which.


Mental Model — A Numbered Sliding Window

flowchart LR
  subgraph LV["one lruvec — struct lru_gen_folio"]
    direction LR
    G3["gen = max_seq % 4<br/>YOUNGEST<br/>(just promoted / faulted)"]
    G2["gen<br/>(active)"]
    G1["gen<br/>(inactive)"]
    G0["gen = min_seq % 4<br/>OLDEST<br/>(eviction drains here)"]
    G3 --> G2 --> G1 --> G0
  end
  AGE["AGING:<br/>find young PTEs,<br/>promote to G3,<br/>then max_seq++"]
  EVI["EVICTION:<br/>reclaim from G0,<br/>then min_seq++<br/>when G0 empties"]
  AGE -->|"produces"| G3
  G0 -->|"consumes"| EVI

The generation window of a single lruvec. What it shows: at most MAX_NR_GENS = 4 generations are live at once. A page’s generation is an index gen in [0, 3] computed as seq % MAX_NR_GENS; the youngest generation is max_seq % 4 and the oldest is min_seq % 4. Aging promotes hot pages into the youngest generation and bumps max_seq (creating a new youngest slot and shifting the window right); eviction reclaims from the oldest generation and bumps min_seq when it empties (shifting the window’s tail right). The insight to take: because max_seq and min_seq only ever increase, demotion is free — a page that is never re-promoted simply finds itself in an older generation as max_seq advances around it, with no per-page list surgery. The classic LRU’s explicit active→inactive demotion is replaced by counter arithmetic.

The two-generation minimum (MIN_NR_GENS) exists for a precise reason that the source comments spell out: the aging “needs to check the accessed bit at least twice before handing this page over to the eviction. The first check takes care of the accessed bit set on the initial fault; the second check makes sure this page hasn’t been used since then. This process, AKA second chance, requires a minimum of two generations” (mmzone.h). The four-generation maximum gives “twice the number of categories of the active/inactive LRU when keeping track of accesses through page tables,” at the cost of order_base_2(MAX_NR_GENS+1) bits in folio->flags.

The window as a state machine

The flowchart above shows the layout; what it cannot show is that the four slots are a ring and that the window’s width is itself a state the two halves of reclaim fight over. Aging widens the window (bumping max_seq); eviction narrows it (bumping min_seq). The kernel’s stated ideal is MIN_NR_GENS + 1 = three live generations, because at exactly MIN_NR_GENS the eviction stalls waiting for the aging (should_run_aging(), mm/vmscan.c):

stateDiagram-v2
  direction LR
  [*] --> W2
  W2: WIDTH 2, equal to MIN_NR_GENS<br/>max_seq - min_seq + 1 == 2<br/>eviction STALLED<br/>scan_folios returns 0
  W3: WIDTH 3, the stated ideal<br/>one old gen to evict<br/>two young gens protected
  W4: WIDTH 4, equal to MAX_NR_GENS<br/>window FULL<br/>aging must roll min_seq first
  W2 --> W3: aging runs try_to_inc_max_seq<br/>walk mm_list, promote young PTEs<br/>then smp_store_release of max_seq+1
  W3 --> W4: aging runs again<br/>hot pages still arriving
  W4 --> W4: inc_max_seq calls inc_min_seq FIRST<br/>bulk-moving the oldest gen forward<br/>4096 folios per lru_lock hold
  W3 --> W2: eviction drains the oldest gen<br/>try_to_inc_min_seq advances min_seq
  W4 --> W3: eviction drains the oldest gen
  note right of W2
    should_run_aging returns true here
    with nr_to_scan set to 0 —
    "completely out of cold folios"
  end note

The generation window as a width-state machine, per inc_max_seq(), try_to_inc_min_seq() and should_run_aging() in mm/vmscan.c at v6.12. What it shows: the window is never narrower than two or wider than four, and the transitions are owned by different actors — aging pushes right at the head, eviction pushes right at the tail. At width 4 the aging cannot simply add a generation; inc_max_seq() first calls inc_min_seq() to physically bulk-move the oldest generation’s folios forward (up to MAX_LRU_BATCH = 4096 at a time, dropping and retaking lru_lock between batches) and only then bumps max_seq. The insight to take: “width 2” is not a healthy resting state — it is the stall state. scan_folios() opens with if (get_nr_gens(lruvec, type) == MIN_NR_GENS) return 0;, so an lruvec sitting at width 2 yields nothing to reclaim until the aging runs. Every design decision about when to age is really a decision about how to avoid living at width 2.


min_seq, max_seq, and the gen Counter

Three counters define the window per lruvec (design doc):

  • max_seq — “the youngest generation number … for both anon and file types as they are aged on an equal footing.” A single value: anonymous and file pages share the youngest generation.
  • min_seq[ANON_AND_FILE] — “the oldest generation numbers … stored … separately for anon and file types as clean file pages can be evicted regardless of swap constraints.” Two values, because file and anonymous reclaim proceed at different rates. “Normally anon and file min_seq are in sync. But if swapping is constrained, e.g., out of swap space, file min_seq is allowed to advance and leave anon min_seq behind” (mmzone.h).
  • The per-page gen counter — packed into folio->flags. “The gen counter in folio->flags stores gen+1 while a page is on one of lrugen->folios[]. Otherwise it stores 0.” So a stored value in [1, MAX_NR_GENS] means “on generation gen,” and 0 means “not on an MGLRU list.” Storing gen+1 rather than gen lets 0 be the unambiguous off-list sentinel.

The generation index used to subscript the per-lruvec arrays is seq % MAX_NR_GENS. Because the sequence numbers are monotonic but the array has only four slots, the window wraps around the four slots like a ring buffer; the kernel uses the sequence numbers (not the wrapped indices) for all ordering comparisons, so wraparound of the index never confuses recency.


The lru_gen_folio Structure

All of a generation window’s state lives in one structure embedded in each lruvec. Here is the verbatim definition from v6.12 (mmzone.h):

struct lru_gen_folio {
	/* the aging increments the youngest generation number */
	unsigned long max_seq;
	/* the eviction increments the oldest generation numbers */
	unsigned long min_seq[ANON_AND_FILE];
	/* the birth time of each generation in jiffies */
	unsigned long timestamps[MAX_NR_GENS];
	/* the multi-gen LRU lists, lazily sorted on eviction */
	struct list_head folios[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES];
	/* the multi-gen LRU sizes, eventually consistent */
	long nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES];
	/* the exponential moving average of refaulted */
	unsigned long avg_refaulted[ANON_AND_FILE][MAX_NR_TIERS];
	/* the exponential moving average of evicted+protected */
	unsigned long avg_total[ANON_AND_FILE][MAX_NR_TIERS];
	/* the first tier doesn't need protection, hence the minus one */
	unsigned long protected[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS - 1];
	/* can be modified without holding the LRU lock */
	atomic_long_t evicted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS];
	atomic_long_t refaulted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS];
	/* whether the multi-gen LRU is enabled */
	bool enabled;
	/* the memcg generation this lru_gen_folio belongs to */
	u8 gen;
	/* the list segment this lru_gen_folio belongs to */
	u8 seg;
	/* per-node lru_gen_folio list for global reclaim */
	struct hlist_nulls_node list;
};

Walking the load-bearing fields:

  • max_seq, min_seq[] — the window counters described above.
  • timestamps[MAX_NR_GENS] — the birth time of each generation in jiffies. This is what min_ttl_ms working-set protection reads (see Multi-Generational LRU): if the oldest generation’s timestamp is younger than min_ttl_ms ago, the lruvec is protected from eviction.
  • folios[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES] — the actual lists of pages, indexed by generation × type × zone. This three-dimensional indexing is why MGLRU can keep anon and file separate (for swappiness balancing) and zone-aware (so a high-order or zone-constrained allocation can reclaim from the right zone). The comment “lazily sorted on eviction” matters: pages are not eagerly re-sorted into the right generation when their gen counter changes; the eviction path sorts a page into its correct generation when it reaches it (see Eviction below).
  • nr_pages[...] — page counts per (gen, type, zone), “eventually consistent” and able to go “transiently negative when reset_batch_size() is pending” because aging batches its updates.
  • avg_refaulted, avg_total, protected, evicted, refaulted — the per-tier statistics that drive the PID feedback loop (next section). evicted and refaulted are atomic_long_t because, as the comment notes, they “can be modified without holding the LRU lock” — refaults are recorded on the fault path, which must not contend on the reclaim lock.
  • gen, seg, list — these belong to the memcg LRU (an LRU of memcgs for global reclaim), not the per-page generations; gen/seg place this lru_gen_folio in the old/young memcg generation and its head/tail/default segment. See the Memcg LRU note below.

The same information as a reference table — this is the structure you will be staring at in a crash dump or a drgn session, so it is worth having the dimensions, the writer and the locking rule for each field in one place:

FieldDeclared typeElements (x86-64, 5 zones)Written byRead bySerialization
max_sequnsigned long1inc_max_seq() onlyeverything, via DEFINE_MAX_SEQ()published with smp_store_release() under lru_lock; read with READ_ONCE()
min_seq[ANON_AND_FILE]unsigned long2inc_min_seq(), try_to_inc_min_seq()eviction, should_run_aging()WRITE_ONCE() under lru_lock
timestamps[MAX_NR_GENS]unsigned long (jiffies)4inc_max_seq() stamps the new slotmin_ttl_ms protection; the debugfs age_in_ms columnlru_lock
folios[gen][type][zone]struct list_head4 x 2 x 5 = 40 listslru_gen_add_folio(), sort_folio(), inc_min_seq()scan_folios()lru_lock
nr_pages[gen][type][zone]long40lru_gen_update_size(), reset_batch_size()should_run_aging(), debugfs“eventually consistent”; may read transiently negative
avg_refaulted[type][tier]unsigned long2 x 4 = 8reset_ctrl_pos(carryover=true)read_ctrl_pos()lru_lock
avg_total[type][tier]unsigned long8reset_ctrl_pos(carryover=true)read_ctrl_pos()lru_lock
protected[hist][type][tier-1]unsigned longNR_HIST_GENS x 2 x 3sort_folio() on the protect pathread_ctrl_pos()lru_lock
evicted[hist][type][tier]atomic_long_tNR_HIST_GENS x 2 x 4evictionread_ctrl_pos()atomics, no lock
refaulted[hist][type][tier]atomic_long_tNR_HIST_GENS x 2 x 4lru_gen_refault() on the fault pathread_ctrl_pos()atomics, no lock
enabledbool1lru_gen_change_state()lru_gen_add_folio() fast pathsstate_mutex + lru_lock
gen, segu81 eachmemcg-LRU operationsshrink_many()see the Memcg LRU section
liststruct hlist_nulls_node1memcg-LRU operationsglobal reclaim traversalRCU

struct lru_gen_folio as a reference table, from include/linux/mmzone.h at v6.12, lines 429-456. What it shows: the structure has three distinct populations — the window counters (cheap, lru_lock-protected scalars), the 40 list heads plus their sizes (the actual page storage), and the per-tier statistics (half of them lock-free atomics). The insight to take: the atomic_long_t on evicted[] and refaulted[] is not a micro-optimization, it is a structural requirement. refaulted[] is incremented by lru_gen_refault(), which runs on the page-fault path — the one path in the kernel that must never queue behind lru_lock, because that lock is held by reclaim, which is exactly what is making the faults happen. Everything the controller needs to read is lru_lock-protected and consistent; everything it needs to write from a fault is an atomic.

The relevant macros, also verbatim (mmzone.h):

#define MIN_NR_GENS		2U
#define MAX_NR_GENS		4U
#define MAX_NR_TIERS		4U

Aging: Producing Young Generations

Aging is triggered when the window is about to run short — “it increments max_seq when max_seq-min_seq+1 approaches MIN_NR_GENS” (design doc). In other words, when only the bare minimum of generations remain, the producer must make a fresh one or eviction will have nothing cold to consume.

The mechanism, step by step (design doc):

  1. Promote hot pages. Aging finds pages “accessed through page tables” and promotes them to the youngest generation. It locates young PTEs two ways — full page-table walks and rmap-driven look-around — both detailed in MGLRU Page Table Walks. For each young PTE found, “the aging clears the accessed bit and updates the gen counter of the page mapped by this PTE to (max_seq%MAX_NR_GENS)+1.” Clearing the accessed bit is the “second chance” reset: the next pass can now tell whether the page is touched again.
  2. Increment max_seq. After a full page-table-walk pass, “it increments max_seq,” creating a brand-new youngest generation slot.
  3. Implicit demotion. Incrementing max_seq is exactly what demotes the cold pages: “the demotion of cold pages happens consequently when it increments max_seq.” No page is moved to do this — the pages that were not re-promoted simply now sit one generation older relative to the advanced max_seq. This is the key efficiency over the classic LRU, which must physically move pages between active and inactive lists.

Because aging promotes only pages it positively observes as young, and demotes the rest for free by advancing the counter, the per-pass cost is proportional to the number of hot pages and the cost of the scan — not to the total resident set. That is the whole point: cheap, periodic, batched aging instead of expensive per-page reclaim-time rmap chases.

When exactly does the aging run? should_run_aging()

“When max_seq-min_seq+1 approaches MIN_NR_GENS” is the design document’s summary, but the code is more specific and more interesting. should_run_aging() (mm/vmscan.c, v6.12) walks every live generation, sums nr_pages[gen][type][zone] into three buckets — total (everything), young (the max_seq generation), old (the generation at max_seq - MIN_NR_GENS) — and then applies four rules in order:

#Predicate (verbatim)ReturnsWhat it means in words
1min_seq[!can_swap] + MIN_NR_GENS > max_seqtrue, with *nr_to_scan = 0The window has collapsed to two. There is nothing cold left to evict; age now or reclaim makes no progress.
2min_seq[!can_swap] + MIN_NR_GENS < max_seqfalseThe window is already wider than the minimum-plus-one. “The aging tries to be lazy to reduce the overhead” — don’t bother.
3young * MIN_NR_GENS > totaltrueMore than 1/2 of all pages sit in the single youngest generation. The hot end is bloated; minting a new generation spreads them out.
4old * (MIN_NR_GENS + 2) < totaltrueFewer than 1/4 of all pages sit in the about-to-be-evicted generation. The cold end is starved; eviction will run dry shortly.
—otherwisefalseDistribution is acceptable.

The aging trigger, decoded. The comment above rules 3 and 4 states the target explicitly: “It’s also ideal to spread pages out evenly, i.e., 1/(MIN_NR_GENS+1) of the total number of pages for each generation. A reasonable range for this average portion is [1/MIN_NR_GENS, 1/(MIN_NR_GENS+2)].” With MIN_NR_GENS = 2 that is a target of 1/3 per generation with an accepted band of 1/4 to 1/2. What it shows: the aging is driven by the shape of the distribution across generations, not by a timer, a watermark, or a page count. The insight to take: rules 3 and 4 are asymmetric on purpose — the comment says “the aging cares about the upper bound of hot pages, while the eviction cares about the lower bound of cold pages.” Rule 3 fires when the producer’s output is piling up; rule 4 fires when the consumer’s input is running out. Both are symptoms of the same disease (the window is degenerating toward width 2) caught from opposite ends.

There is one more gate, in the caller. get_nr_to_scan() runs should_run_aging() and then:

/* try to get away with not aging at the default priority */
if (!success || sc->priority == DEF_PRIORITY)
        return nr_to_scan >> sc->priority;
 
/* stop scanning this lruvec as it's low on cold folios */
return try_to_inc_max_seq(lruvec, max_seq, can_swap, false) ? -1 : 0;

Read that carefully, because the comment is the whole policy: at DEF_PRIORITY (12, the first and gentlest reclaim pass) the aging is skipped even when should_run_aging() said yes. The kernel first tries to satisfy the allocation out of whatever cold pages already exist, scaled down by >> sc->priority. Only when the priority has been ratcheted down — i.e. the easy pass failed and reclaim is getting serious — does it pay for a full mm_list walk. Aging is a cost the kernel defers as long as it can; see Direct Reclaim and kswapd and Background Reclaim for who ratchets sc->priority and when.

flowchart TD
  A["get_nr_to_scan(lruvec, sc, can_swap)"] --> B{"memcg below memory.min?"}
  B -->|yes| Z1["return -1<br/>(skip this lruvec entirely)"]
  B -->|no| C["should_run_aging():<br/>sum nr_pages[] into total / young / old"]
  C --> D{"window width == MIN_NR_GENS?<br/>min_seq + 2 &gt; max_seq"}
  D -->|yes| E["nr_to_scan = 0, success = true"]
  D -->|no| F{"window wider than<br/>MIN_NR_GENS + 1?"}
  F -->|yes| G["success = false<br/>(be lazy)"]
  F -->|no| H{"young &gt; total/2<br/>OR old &lt; total/4 ?"}
  H -->|yes| I["success = true"]
  H -->|no| G
  E --> J{"memcg offlined?"}
  I --> J
  G --> J
  J -->|yes| Z2["return nr_to_scan<br/>(scrape it dry)"]
  J -->|no| K{"success == false<br/>OR sc-&gt;priority == DEF_PRIORITY?"}
  K -->|yes| Z3["return nr_to_scan &gt;&gt; sc-&gt;priority<br/>DO NOT AGE — evict what is already cold"]
  K -->|no| L["try_to_inc_max_seq()<br/>the real aging pass"]
  L -->|succeeded| Z4["return -1<br/>stop scanning this lruvec this round"]
  L -->|raced / failed| Z5["return 0"]

The full aging-decision path, from get_nr_to_scan() and should_run_aging() in mm/vmscan.c at v6.12. What it shows: three separate escape hatches fire before any page table is touched — the memcg protection check, the laziness rule, and the DEF_PRIORITY deferral. The insight to take: on a healthy machine the aging almost never runs, because sc->priority == DEF_PRIORITY short-circuits it on every background reclaim pass. Aging is a second-line response, reached only when the cheap pass has failed. That is why MGLRU’s CPU cost does not scale with resident set size on an idle-ish system: the expensive part is gated behind evidence that the cheap part did not work.

Minting the generation: inside inc_max_seq()

try_to_inc_max_seq() does the scanning (delegated to MGLRU Page Table Walks); inc_max_seq() does the bookkeeping, and it is worth tracing because three non-obvious things happen under one lru_lock acquisition:

  1. Make room if the window is full. For each type, if (get_nr_gens(lruvec, type) != MAX_NR_GENS) continue; — otherwise call inc_min_seq(), which physically walks the oldest generation’s 5 zone lists and list_move_tail()s every folio into the next generation, up to MAX_LRU_BATCH = 4096 folios. If it hits that cap it returns false, and inc_max_seq() drops lru_lock, calls cond_resched(), and restarts from the top. On a machine with a large oldest generation this loop can run many times; it is the one place in the aging where cost genuinely is proportional to page count.
  2. Repaint the active/inactive projection. MGLRU still has to report nr_active_file and friends, and lru_gen_is_active() defines “active” as the two youngest generations. Advancing max_seq therefore silently reclassifies a whole generation from active to inactive, so inc_max_seq() computes delta = nr_pages[max_seq-1] - nr_pages[max_seq+1] per type per zone and applies __update_lru_size() twice — once positive on the inactive list, once negative on the active list. The comment explains why both sides of max_seq are needed: “max_seq+1 can overlap with min_seq[LRU_GEN_ANON] if swapping is constrained. And if they do overlap, cold/hot inversion happens.”
  3. Roll the controller and stamp the clock. reset_ctrl_pos(lruvec, type, false) folds the just-finished generation’s refault counters into the exponential moving averages; WRITE_ONCE(lrugen->timestamps[next], jiffies) stamps the new generation’s birth time (this is what min_ttl_ms and the debugfs age_in_ms column read); and only then smp_store_release(&lrugen->max_seq, lrugen->max_seq + 1) publishes the new width. The release barrier matters: readers outside the lock use READ_ONCE(max_seq) and must not observe the new max_seq before the timestamps[] and size updates that justify it.

Aging and Eviction, Side by Side

The two halves are a producer–consumer pair over the same sliding window, and almost every property of MGLRU falls out of the asymmetry between them:

Aging (the producer)Eviction (the consumer)
Entry pointtry_to_inc_max_seq() → walk_mm() → inc_max_seq()evict_folios() → isolate_folios() → scan_folios()
Moves which countermax_seq (head of the window)min_seq[type] (tail of the window)
Triggered byshould_run_aging() distribution rules, and sc->priority < DEF_PRIORITYevery reclaim pass; it is the default work
Inputprocess page tables (lruvec_memcg()->mm_list) and rmap look-aroundthe folio lists at min_seq % MAX_NR_GENS
Per-folio work on the common pathnone — a folio that is not young is never touchedisolate, sort, or reclaim, one folio at a time
How a folio moves downimplicitly: max_seq advances past it, no list surgeryexplicitly: sort_folio() calls list_move()
How a folio moves upfolio_update_gen() writes a new gen into folio->flags, batched via update_batch_size()sort_folio() promotes protected folios with folio_inc_gen()
Lock disciplinetakes lru_lock once per batch (reset_batch_size()), plus the PTE/PMD spinlocks during the walktakes lru_lock around each MAX_LRU_BATCH isolation run
Feedback it producesinserts dense PMDs into the Bloom filter (carried to seq+1)records evicted[]/protected[], and refaults record refaulted[]
Feedback it consumesthe Bloom filter written by the previous generation and by lru_gen_look_around()read_ctrl_pos() / positive_ctrl_err() over the tier statistics
What it costs when it goes wronga wasted page-table sweep that finds no young PTEsa premature eviction, paid back as a refault and a major fault

Producer and consumer, compared. What it shows: the two halves touch completely different data structures and take completely different locks, which is why they can be reasoned about — and tuned — separately. The insight to take: the “how a folio moves down” row is the single most important line in this note. In the classic LRU, demotion is shrink_active_list() physically walking the active list and moving folios; that scan is counted in pgrefill and the moves in pgdeactivate. In MGLRU, demotion is one increment of one unsigned long. That is why, on the Fedora host measured below, pgdeactivate is exactly zero after 4.6 million reclaimed pages.


Eviction: Consuming Old Generations

Eviction is the consumer and lives in the reclaim path; the generation-bookkeeping parts relevant here are:

  • It “increments min_seq when lrugen->folios[] indexed by min_seq%MAX_NR_GENS becomes empty” — i.e. once the oldest generation is fully drained, the tail of the window slides forward (design doc).
  • It performs the lazy sort promised by the folios[] comment: “the eviction sorts a page according to its gen counter if the aging has found this page accessed through page tables and updated its gen counter.” So a page the aging promoted (by writing a younger gen into folio->flags) but did not physically move is moved to its correct generation only when eviction reaches it — deferring the list work to the moment it is needed.
  • It can give a frequently-used file page another reprieve: “It also moves a page to the next generation, i.e., min_seq+1, if this page was accessed multiple times through file descriptors and the feedback loop has detected outlying refaults from the tier this page is in.”

To choose which type to evict, eviction “first compares min_seq[] to select the older type. If both types are equally old, it selects the one whose first tier has a lower refault percentage. The first tier contains single-use unmapped clean pages, which are the best bet” (design doc) — clean file pages can be dropped with no I/O.

sort_folio() — the six-way decision at the tail of the window

The design document’s three bullets compress a function that actually has six outcomes. sort_folio() (mm/vmscan.c, v6.12) is called on every folio scan_folios() pulls off the oldest generation’s list, and returns true if it handled the folio (put it somewhere else) or false if the folio should proceed to isolation and reclaim. The order of the checks is load-bearing:

OrderConditionActionWhere the folio ends up
1!folio_evictable(folio)lru_gen_del_folio(), set PG_unevictable, re-add via lruvec_add_folio(), count UNEVICTABLE_PGCULLEDoff MGLRU entirely, on the unevictable list
2gen != lru_gen_from_seq(min_seq[type]) — promotedlist_move() to the head of its actual generation’s listthe lazy sort: the aging wrote a younger gen into folio->flags without moving it; this is where the move finally happens
3tier > tier_idx || refs == BIT(LRU_REFS_WIDTH) — protectedfolio_inc_gen(), list_move_tail() to min_seq+1, and protected[hist][type][tier-1] += deltaone generation younger; this is the refault loop granting a reprieve
4!folio_test_lru(folio) || zone > sc->reclaim_idx — ineligiblefolio_inc_gen(), list_move_tail()one generation younger, for a reason that has nothing to do with heat — the caller is reclaiming for a lower zone than this folio lives in
5locked, under writeback, or a dirty file folio — waiting for I/Ofolio_inc_gen(reclaiming=true), list_move() to the headone generation younger, at the head so it is revisited soon; the reclaiming=true argument additionally sets PG_reclaim, which folio_end_writeback() uses to rotate the folio when the I/O completes
6none of the abovereturn falsethe caller isolates it and hands it to shrink_folio_list()

The eviction’s per-folio sort, from sort_folio() at v6.12. What it shows: four of the six outcomes move the folio to a younger generation, and only one of those four (case 3) is a genuine heat judgement. Cases 4 and 5 are mechanical deferrals. The insight to take: folio_inc_gen() is doing double duty as both “this page is hotter than we thought” and “we cannot deal with this page right now” — which means a generation’s population is not purely a statement about recency. A machine doing heavy dirty-page writeback will see folios cycling up one generation at a time through case 5 with no access having occurred at all. When the generation histogram looks implausibly hot under writeback pressure, this is why.

The tier_idx in case 3 is not a fixed threshold — it is recomputed per scan by get_tier_idx(), which is the subject of the next section.

Every generation move resets the tier. This is the least intuitive fact in the whole design and it is easy to get backwards. Both folio_update_gen() (used by the aging) and folio_inc_gen() (used by sort_folio()) build their new flags word the same way:

new_flags = old_flags & ~(LRU_GEN_MASK | LRU_REFS_MASK | LRU_REFS_FLAGS);
new_flags |= (new_gen + 1UL) << LRU_GEN_PGOFF;

LRU_REFS_FLAGS is BIT(PG_referenced) | BIT(PG_workingset) and LRU_REFS_MASK is the spare counter bits — so writing a new generation number clears the access counter and both reference flags in the same try_cmpxchg. A folio that earns protection at tier 2 re-enters the window at tier 0. The consequence: tiers measure access frequency within one trip through the generation window, not over the folio’s lifetime. To stay protected, a folio has to be re-accessed on every lap. That is what stops a page that was hot once from coasting forever on an old reputation, and it is why protected[] is a per-generation counter rather than a per-folio flag.

The one exception is in folio_update_gen() itself: if !(old_flags & LRU_GEN_MASK) — the folio has already been isolated by lru_gen_del_folio() and is off the lists — it sets PG_referenced instead and leaves the generation alone, “for shrink_folio_list()”. A folio the aging finds young while it is in flight toward reclaim is not promoted (there is no list to promote it onto); it is merely marked referenced so the reclaimer rotates it back instead of dropping it.

flowchart TD
  S["scan_folios(): pop folio from<br/>folios[min_seq%4][type][zone]"] --> A{"folio_evictable()?"}
  A -->|no| R1["cull to the unevictable list<br/>UNEVICTABLE_PGCULLED++"]
  A -->|yes| B{"gen != min_seq's gen?<br/>(aging promoted it)"}
  B -->|yes| R2["LAZY SORT<br/>list_move() to its real generation"]
  B -->|no| C{"tier &gt; tier_idx<br/>or refs saturated?"}
  C -->|yes| R3["PROTECT<br/>folio_inc_gen() to min_seq+1<br/>protected[hist][type][tier-1] += n"]
  C -->|no| D{"off-LRU, or<br/>zone &gt; sc-&gt;reclaim_idx?"}
  D -->|yes| R4["DEFER (ineligible)<br/>folio_inc_gen(), move to tail"]
  D -->|no| E{"locked / writeback /<br/>dirty file folio?"}
  E -->|yes| R5["DEFER (I/O)<br/>folio_inc_gen(reclaiming=true)<br/>move to HEAD, also sets PG_reclaim"]
  E -->|no| R6["RECLAIM<br/>isolate_folio() then<br/>shrink_folio_list()"]
  R6 --> F{"reclaimed?"}
  F -->|yes| G["evicted[hist][type][tier] += n<br/>PGSTEAL_* += n"]
  F -->|"no — rotated back"| H["returned to the list;<br/>retried with skip_retry"]

The eviction’s decision cascade for one folio. What it shows: the path to actual reclaim is guarded by five prior tests, three of which push the folio back up a generation. The insight to take: compare this with the classic shrink_inactive_list() → shrink_folio_list() path in The LRU Lists. The classic scanner makes its keep-or-drop decision by consulting the rmap for the accessed bit at this exact moment; MGLRU has already done that work during the aging, so sort_folio() reads only folio->flags and the pre-computed tier_idx. That is the concrete meaning of “moving the cost out of the reclaim path.”


Tiers and Refault Feedback

Generations capture recency; tiers capture frequency, orthogonally, within a generation. The source comment is precise: “Each generation is divided into multiple tiers. A page accessed N times through file descriptors is in tier order_base_2(N)” (mmzone.h). So tier 0 is pages accessed 0–1 times, tier 1 is 2–3 times, tier 2 is 4–7 times, and so on, with MAX_NR_TIERS = 4 tiers.

How a tier is actually computed, bit by bit

The header comment says order_base_2(N); the code takes a detour worth following, because it explains why “0 and 1 accesses” collapse into the same tier. Two inline helpers in include/linux/mm_inline.h at v6.12 do the work:

static inline int lru_tier_from_refs(int refs)
{
	/* see the comment in folio_lru_refs() */
	return order_base_2(refs + 1);
}
 
static inline int folio_lru_refs(struct folio *folio)
{
	unsigned long flags = READ_ONCE(folio->flags);
	bool workingset = flags & BIT(PG_workingset);
 
	/*
	 * Return the number of accesses beyond PG_referenced, i.e., N-1 if the
	 * total number of accesses is N>1, since N=0,1 both map to the first
	 * tier. lru_tier_from_refs() will account for this off-by-one.
	 */
	return ((flags & LRU_REFS_MASK) >> LRU_REFS_PGOFF) + workingset;
}

refs is therefore not the access count — it is the access count minus one, reconstructed from LRU_REFS_WIDTH spare flag bits plus the PG_workingset bit. LRU_REFS_WIDTH is min(__LRU_REFS_WIDTH, ...whatever is left in page->flags...) (include/linux/page-flags-layout.h, v6.12, lines 109-112), which on a normal x86-64 SPARSEMEM_VMEMMAP build is 2 bits, matching the header’s “uses MAX_NR_TIERS-2 spare bits.” Working the arithmetic through:

Accesses N through file descriptorsPG_referencedPG_workingsetLRU_REFS bitsrefsTier = order_base_2(refs+1)Meaning
0clearclear0000never touched since it entered the list
1setclear0000touched once — indistinguishable from never, by design
2setset0011first evidence of re-use
3setset0122
4setset1032
5setset1143saturated: refs == BIT(LRU_REFS_WIDTH)
≥ 6setset1143counter pegged; no further discrimination

Tier assignment on a typical x86-64 build with LRU_REFS_WIDTH = 2. What it shows: the frequency counter saturates after five accesses, and the first two accesses are deliberately conflated. The insight to take: the collapse of N=0 and N=1 into tier 0 is the whole anti-streaming defence in one line. A cat big.log > /dev/null touches every page exactly once, so every one of those pages lands in tier 0 and is evicted first, no matter how recently it was read. A page that is read twice immediately separates itself into tier 1 and becomes eligible for protection. Saturation at refs == BIT(LRU_REFS_WIDTH) is also explicitly special-cased in sort_folio() (tier > tier_idx || refs == BIT(LRU_REFS_WIDTH)), so a maximally-hot folio is protected unconditionally rather than having to beat a threshold.

One more subtlety, in isolate_folio(): if (!folio_test_referenced(folio)) set_mask_bits(&folio->flags, LRU_REFS_MASK | LRU_REFS_FLAGS, 0); — a folio that reaches isolation with PG_referenced clear has its entire reference history wiped before it is handed to shrink_folio_list(). The counter is not a lifetime statistic; it is a rolling window that resets whenever the folio goes all the way round without being touched.

The decisive property of tiers is that they are cheap to move between. “Unlike generations, tiers do not have dedicated lrugen->folios[]. In contrast to moving across generations, which requires the LRU lock, moving across tiers only involves atomic operations on folio->flags and therefore has a negligible cost in the buffered access path” (mmzone.h). A page records access-frequency promotions in its flag bits (PG_referenced, PG_workingset) without ever touching the contended reclaim lock — only generation moves need the lock.

Tiers feed a PID-controller refault loop. The kernel tracks, per tier per type, how many pages it evicted and how many of those refaulted (were faulted back in soon after — proof the eviction was premature). It keeps exponential moving averages (avg_refaulted, avg_total). In the eviction path, “comparisons of refaulted/(evicted+protected) from the first tier and the rest infer whether pages accessed multiple times through file descriptors are statistically hot and thus worth protecting” (mmzone.h). Tier 0 (single-use pages) is the baseline; if a higher tier refaults more than tier 0, its pages are statistically hot and get protected (moved to a younger generation rather than evicted). The protected[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS - 1] array tracks exactly this — note the - 1: “the first tier doesn’t need protection, hence the minus one.” This is precisely the defense against the classic LRU’s failure mode of one-off I/O flooding out the working set: a burst of single-use reads stays in tier 0 and is evicted first, while genuinely re-read pages climb tiers and earn protection. LWN described this design as computing “per-tier refault rates and protecting higher-tier pages from reclaim if their refault rate exceeds tier 0’s rate” (LWN 856931).

The PID controller deliberately uses generations as its time domain, not the wall clock, “because a CPU can scan pages at different rates under varying memory pressure[, and] it calculates a moving average for each new generation to avoid being permanently locked in a suboptimal state” (design doc). The net effect is to “balance refault percentages between anon and file types proportional to the swappiness level” — MGLRU’s principled replacement for the blunt swappiness arbitration of the classic scanner (see Swappiness and Reclaim Balance).

The controller’s arithmetic, symbol by symbol

“PID controller” is the design document’s framing, but the implementation is compact enough to read in full. Three functions in mm/vmscan.c at v6.12 do everything.

Step 1 — take a reading. read_ctrl_pos(lruvec, type, tier, gain, &pos) fills a three-field struct ctrl_pos:

pos->refaulted = lrugen->avg_refaulted[type][tier] +
                 atomic_long_read(&lrugen->refaulted[hist][type][tier]);
pos->total     = lrugen->avg_total[type][tier] +
                 atomic_long_read(&lrugen->evicted[hist][type][tier]);
if (tier)
        pos->total += lrugen->protected[hist][type][tier - 1];
pos->gain = gain;

Symbol by symbol: refaulted is smoothed history plus current generation’s refaults; total is smoothed history plus current evictions, plus — for any tier above zero — the pages that were protected instead of evicted. Adding protected[] into the denominator is what makes the comparison fair: a tier that was spared eviction should not look good merely because it was never tested. gain is a caller-supplied weight, discussed in step 3.

Step 2 — compare two readings. positive_ctrl_err(sp, pv) asks whether the process variable pv has a lower weighted refault rate than the set point sp:

return pv->refaulted < MIN_LRU_BATCH ||
       pv->refaulted * (sp->total + MIN_LRU_BATCH) * sp->gain <=
       (sp->refaulted + 1) * pv->total * pv->gain;

This is pv.refaulted / pv.total <= sp.refaulted / sp.total, cross-multiplied into integers, with three defensive adjustments. The + MIN_LRU_BATCH (64) on sp->total and the + 1 on sp->refaulted avoid a division-by-zero-equivalent and damp the comparison when the set point has almost no history. The short-circuit pv->refaulted < MIN_LRU_BATCH says: if this tier has produced fewer than 64 refaults in total, do not believe the ratio at all — treat it as good. The gain factors are the swappiness/margin weights multiplied into each side.

Step 3 — use it twice, with different gains. The same predicate answers two different questions depending on the gains passed in:

Callersp (set point)pv (process variable)GainsQuestion answered
get_tier_idx(lruvec, type)tier 0 of this typetier 1, 2, 3 in turnsp=1, pv=2“How far up the tiers do I have to go before a tier refaults badly enough to be worth protecting?” Returns the last tier that is safe to evict.
get_type_to_scan(lruvec, swappiness, &tier_idx)tier 0 of the other typetier 0 of this type, then higher tiersswappiness vs MAX_SWAPPINESS - swappiness“Anon or file?” — and then the same tier scan within the winner.

The get_tier_idx() comment names the margin explicitly: “To leave a margin for fluctuations, use a larger gain factor (1:2). This value is chosen because any other tier would have at least twice as many refaults as the first tier.” So a higher tier must refault at more than twice tier 0’s rate before MGLRU will start protecting it — deliberate hysteresis, so a brief refault spike does not flip the policy.

In get_type_to_scan() the gains are gain[LRU_GEN_ANON] = swappiness and gain[LRU_GEN_FILE] = MAX_SWAPPINESS - swappiness. This is where swappiness re-enters MGLRU — not as a scan-ratio the way get_scan_count() uses it in the classic path, but as a weight on the refault comparison. Raising swappiness does not make the kernel scan more anonymous pages; it makes anonymous refaults count for less when deciding which type is cheaper to evict. That difference in meaning is the mechanical reason behind the 2026 Android complaint that “adjusting the swappiness knob does not fix” MGLRU’s anonymous-page retention (Corbet, LWN, 2026-03-05); see Swappiness and Reclaim Balance.

Step 4 — decay. reset_ctrl_pos(lruvec, type, carryover=true) runs when min_seq advances, and folds the finished generation into the averages with a plain halving:

sum = lrugen->avg_refaulted[type][tier] + atomic_long_read(&lrugen->refaulted[hist][type][tier]);
WRITE_ONCE(lrugen->avg_refaulted[type][tier], sum / 2);

That is an exponential moving average with α = ½ — the cheapest possible one, no multiply, no divide by a variable. Its time domain is generations, not seconds, which is the design document’s stated reason: “a CPU can scan pages at different rates under varying memory pressure[, so] it calculates a moving average for each new generation to avoid being permanently locked in a suboptimal state.”

flowchart LR
  subgraph WRITE["Writers — who feeds the loop"]
    EV["eviction<br/>evicted[hist][type][tier] += n"]
    PR["sort_folio() protect path<br/>protected[hist][type][tier-1] += n"]
    RF["lru_gen_refault()<br/>on the PAGE FAULT path<br/>refaulted[hist][type][tier] += n"]
  end
  subgraph READ["Readers — who uses it"]
    RC["read_ctrl_pos(type, tier, gain)<br/>refaulted = avg + current<br/>total = avg + evicted + protected"]
    PE["positive_ctrl_err(sp, pv)<br/>cross-multiplied ratio test<br/>with MIN_LRU_BATCH noise floor"]
    GT["get_tier_idx()<br/>gain 1:2 margin<br/>-&gt; last tier safe to evict"]
    GY["get_type_to_scan()<br/>gain swappiness : 200-swappiness<br/>-&gt; anon or file"]
  end
  DEC["reset_ctrl_pos(carryover=true)<br/>avg = (avg + current) / 2<br/>runs when min_seq advances"]
  EV --> RC
  PR --> RC
  RF --> RC
  RC --> PE
  PE --> GT
  PE --> GY
  GT --> SF["sort_folio(): tier &gt; tier_idx<br/>-&gt; PROTECT instead of evict"]
  GY --> SF
  SF -.->|"closes the loop"| PR
  RC --> DEC
  DEC -.->|"halved into avg_*"| RC

The refault feedback loop end to end. What it shows: three writers (two in reclaim, one on the fault path), one reading function, one comparison predicate, two policy decisions, and a decay step tied to generation turnover rather than wall clock. The insight to take: the loop is genuinely closed — protecting a tier increases protected[], which increases that tier’s total, which lowers its measured refault rate, which makes it less likely to be protected next time. The controller cannot run away in either direction, and it does so without a single floating-point operation or tunable coefficient.


A Folio’s Life Across Generations and Tiers

Generations and tiers are orthogonal axes and it is easy to lose track of which one is moving. The following trace follows a single file-backed folio from first read to refault, naming the function that touches it at each step. It is the concrete answer to “what actually happens to a page”:

sequenceDiagram
    autonumber
    participant APP as application
    participant PC as page cache / fault path
    participant FL as folio->flags
    participant LG as lrugen->folios[gen][type][zone]
    participant AG as aging (try_to_inc_max_seq)
    participant EV as eviction (sort_folio / evict_folios)
    participant WS as workingset shadow entry

    APP->>PC: first read() of a file page
    PC->>FL: folio_add_lru() sets gen = max_seq%4 + 1
    PC->>LG: added to the YOUNGEST generation, tier 0
    Note over FL,LG: N=0 accesses recorded. PG_referenced clear.
    APP->>PC: second read() of the same page
    PC->>FL: folio_mark_accessed() sets PG_referenced
    Note over FL: still tier 0 — N=0 and N=1 both map to tier 0
    APP->>PC: third read()
    PC->>FL: PG_workingset set, refs becomes 1, so TIER 1
    Note over FL,LG: tier change is an atomic flag write.<br/>NO list move, NO lru_lock.
    AG->>AG: should_run_aging says the window is at width 2
    AG->>FL: max_seq incremented. The folio did NOT move,<br/>but is now one generation older
    Note over LG: implicit demotion, zero per-folio work
    AG->>AG: max_seq incremented again
    LG->>EV: folio now sits in the min_seq generation, scan_folios pops it
    EV->>EV: get_tier_idx returns tier_idx of 0 this round
    EV->>FL: tier 1 beats tier_idx 0, so PROTECT
    EV->>LG: folio_inc_gen to min_seq+1, protected[hist][file][0] grows
    Note over FL,EV: one reprieve granted by the refault loop —<br/>but folio_inc_gen ALSO clears LRU_REFS_MASK<br/>and LRU_REFS_FLAGS, so the folio is back at TIER 0
    AG->>FL: two more max_seq increments with no access
    LG->>EV: back at min_seq at tier 0, no protection this time
    EV->>EV: isolate_folio wipes the refs history if PG_referenced is clear
    EV->>WS: shrink_folio_list reclaims it,<br/>lru_gen_eviction stores a gen and tier token in the shadow
    EV->>EV: evicted[hist][file][tier] grows, PGSTEAL_FILE grows
    APP->>PC: read the same offset again, a MAJOR FAULT
    PC->>WS: workingset_refault decodes the shadow token
    WS->>FL: lru_gen_refault adds to refaulted[hist][file][tier]
    WS->>LG: folio re-inserted into the youngest generation
    Note over WS,LG: the refault is now evidence in the controller,<br/>so next time this tier is likelier to be protected

One file folio’s complete round trip, traced through folio_add_lru(), folio_mark_accessed(), inc_max_seq(), sort_folio(), isolate_folio(), lru_gen_eviction() and lru_gen_refault() at v6.12. What it shows: two independent clocks are running. The tier advances only on accesses through file descriptors and costs one atomic flag write. The generation advances only when the aging bumps max_seq, and costs the folio nothing at all — the folio literally does not move. The insight to take: steps 8 and 10 are the crux of MGLRU. Two generations went by and the folio aged by two steps without a single write to it, a single list operation, or a single lock acquisition. Compare The LRU Lists, where the equivalent ageing requires shrink_active_list() to walk the folio, consult the rmap, and physically list_move() it — and where the deactivation shows up in pgdeactivate. Note also step 20: the refault does not merely reload the page, it feeds the controller. A page that thrashes teaches MGLRU to stop evicting its tier.

Uncertain

Verify: the exact shape of the shadow token stored by lru_gen_eviction() and decoded by lru_gen_refault() (steps 18 and 20), and whether the tier recorded in the shadow is the tier at eviction time or after isolate_folio()’s reference-history wipe. Reason: both functions live in mm/workingset.c, which was not read during this write-up — the trace above is reconstructed from their call sites in mm/vmscan.c and from the refaulted[] field comments. To resolve: read lru_gen_eviction(), lru_gen_test_recent() and lru_gen_refault() in mm/workingset.c at v6.12. The generation and tier semantics described everywhere else in this note come from source that was read. uncertain


The Memcg LRU (Generations of Cgroups)

A second, easily-confused use of the word “generation” appears in lru_gen_folio.gen/seg and in struct lru_gen_memcg: the memcg LRU, “a per-node LRU of memcgs … also an LRU of LRUs” (design doc). For global reclaim, the kernel needs to pick which control group to reclaim from; the memcg LRU orders memcgs into just two generations (old and young) per node, sharded across MEMCG_NR_BINS (8) bins for parallelism. By analogy to the page LRU: young/old memcgs are the counterparts to active/inactive, and incrementing a memcg’s max_seq is the counterpart to activation. The payoff is scalability: traversing memcgs during global reclaim improves “the best-case complexity from O(n) to O(1)” while keeping worst-case O(n), “therefore, on average, … a sublinear complexity.” The relevant macros are MEMCG_NR_GENS = 3 (three slots, only two valid generations — the third prevents a stale lockless read from wrapping a value of seq-1 into “young”) and MEMCG_NR_BINS = 8 (mmzone.h). This applies only to global reclaim; targeted memcg reclaim still uses mem_cgroup_iter().

Uncertain

Verify the exact promotion/demotion trigger events for the memcg LRU (the MEMCG_LRU_HEAD/TAIL/OLD/YOUNG operations and the seven events that trigger them) against the running 6.12/6.18 source if a reader needs that level of detail — they are documented in the mmzone.h comment block but were not independently traced through mm/vmscan.c during this write-up. Reason: cited from the header comment only. To resolve: read shrink_many() / shrink_node_memcgs() in mm/vmscan.c at the v6.12 tag. uncertain


Inspecting Generations at Runtime

Multi-Generational LRU covers the debugfs interface as an operations tool. This section covers it as a window onto the bookkeeping described above — what each column is, and which of the constants in the opening table determines whether a given command is legal.

Reading lru_gen: the generation histogram

lru_gen_seq_show() (mm/vmscan.c, v6.12) prints, per memcg and per node, one line per live generation:

memcg     0 /
 node     0
        4192      31200    1048576     262144
        4193      12400     524288     198432
        4194       2100      65536      44100
ColumnPrinted fromMeaning
1seqthe raw sequence number, not the wrapped gen index — so it increases forever and you can watch it advance
2jiffies_to_msecs(jiffies - lrugen->timestamps[gen])age_in_ms: how long ago this generation was born. This is timestamps[] from the struct table above, and the same value min_ttl_ms compares against
3sum over zones of nr_pages[gen][ANON][zone]anonymous pages in this generation, clamped at zero per zone
4sum over zones of nr_pages[gen][FILE][zone]file pages in this generation

The rows are ordered oldest-first: the first row is min_seq[LRU_GEN_ANON] and the last is max_seq. The admin guide states the interpretation directly — “min_gen_nr contains the coldest pages and max_gen_nr contains the hottest pages, since age_in_ms of the former is the largest and that of the latter is the smallest” — and stresses that “the histograms are noncumulative,” i.e. column 3 is pages in this generation, not this generation and younger (Documentation/admin-guide/mm/multigen_lru.rst, v6.12). Since MAX_NR_GENS = 4, there are at most four rows per node — which is exactly the “MAX_NR_GENS decides the number of bins for each histogram” sentence in the admin guide.

lru_gen_full is the same output plus, for every generation, one line per tier and one line of mm_walk statistics. The trailing letter on each number is the legend, and it is not documented outside the source:

SuffixEmitted whenField
R T xthe row is max_seqavg_refaulted, avg_total, unused — the smoothed history
r e pthe row is min_seq[type], or NR_HIST_GENS > 1refaulted[], evicted[], protected[] — the raw current counters
x x xotherwiseno data retained for this generation
T Y F Aseq == max_seq and NR_HIST_GENS == 1MM_LEAF_TOTAL, MM_LEAF_YOUNG, MM_NONLEAF_FOUND, MM_NONLEAF_ADDED
t y f aseq != max_seq and NR_HIST_GENS > 1the same four, for a historical generation

The lru_gen_full column legend, decoded from lru_gen_seq_show_full() at v6.12. What it shows: uppercase means “smoothed/current-generation”, lowercase means “historical”, and x means “this build does not keep it”. The insight to take: without CONFIG_LRU_GEN_STATS=y, NR_HIST_GENS is 1, so every generation except min_seq and max_seq prints xxx — the historical tier statistics simply are not stored. If you are trying to debug the refault controller and every row is x, the answer is a rebuild, not a different command. The T/Y pair is the diagnostic referenced in MGLRU Page Table Walks: MM_LEAF_YOUNG / MM_LEAF_TOTAL is the yield of the page-table walk.

The two verbs are validated against MIN_NR_GENS and MAX_NR_GENS, so the command grammar is a direct consequence of the window:

VerbGrammarGuard in run_aging() / run_eviction()Failure
+ (age)+ memcg_id node_id max_gen_nr [can_swap [force_scan]]seq < max_seqreturns 0 silently — you asked for a generation that already exists
+seq > max_seq-EINVAL — you can only ask for the next generation, never skip ahead
+!force_scan && min_seq[!can_swap] + MAX_NR_GENS - 1 <= max_seq-ERANGE — the window is already at width 3 or 4; without force_scan the kernel refuses to widen it further
- (evict)- memcg_id node_id min_gen_nr [swappiness [nr_to_reclaim]]seq + MIN_NR_GENS > max_seq-EINVAL — this is the code behind the admin guide’s “min_gen_nr should be less than max_gen_nr-1, since max_gen_nr and max_gen_nr-1 are not fully aged (equivalent to the active list) and therefore cannot be evicted”
-loop exits when seq < min_seq[!swappiness], or nr_reclaimed >= nr_to_reclaim, or evict_folios() returns 0—
-signal_pending(current)-EINTR — the write is interruptible, so a long proactive reclaim can be ^C’d

The debugfs verbs, validated. Grammar from the v6.12 admin guide; guards read from run_aging() and run_eviction() in mm/vmscan.c. Multiple commands may be concatenated on one write with , or ; delimiters. What it shows: the -ERANGE and -EINVAL conditions are MAX_NR_GENS and MIN_NR_GENS respectively, wearing an ABI costume. The insight to take: force_scan defaults to 1 for the + verb, which is why the working-set-estimation workflow works at all — without it, run_aging() would -ERANGE on exactly the healthy machines a job scheduler most wants to measure. The admin guide’s warning that setting force_scan=0 “employs heuristics to reduce the overhead, which is likely to reduce the coverage as well” is the !walk->force_scan branches inside walk_pmd_range(), which is where the Bloom filter gets consulted; see MGLRU Page Table Walks.

Uncertain

Verify: the swappiness argument’s valid range for the - verb. At v6.12 the admin guide says only that it “overrides the default value in /proc/sys/vm/swappiness”. At v7.2 the same paragraph reads “the valid range is [0-200, max], with max being exclusively used for the reclamation of anonymous memory” — the SWAPPINESS_ANON_ONLY addition, existence-checked to v6.16 (absent from include/linux/swap.h at v6.15, present at v6.16). Reason: the v6.12 documentation does not state a range at all, and the v6.12 MAX_SWAPPINESS value was not read during this write-up. To resolve: read MAX_SWAPPINESS and the - verb’s argument parsing in mm/vmscan.c at v6.12 and confirm whether 200 or 100 is the v6.12 ceiling. uncertain


Verified on This Machine

Everything above is read from v6.12 source. This section is a live check on a different tree — kernel 7.1.13-200.fc44 (Fedora 44, x86-64) — and it is included because a distribution kernel is what a reader will actually have in front of them. Where the two trees are known to agree (the design document and the LRU_GEN Kconfig block are byte-identical from v6.12 through v7.2, verified above), a 7.1 observation is evidence about the v6.12 mechanism too; where they may differ, that is called out.

Build-time configuration, from /boot/config-7.1.13-200.fc44.x86_64:

CONFIG_ARCH_HAS_HW_PTE_YOUNG=y
CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG=y
CONFIG_LRU_GEN=y
CONFIG_LRU_GEN_ENABLED=y
# CONFIG_LRU_GEN_STATS is not set
CONFIG_LRU_GEN_WALKS_MMU=y

So MGLRU is compiled in and default-on, the architecture provides both the leaf and the non-leaf accessed bit, and — importantly for the table above — NR_HIST_GENS is 1, so this kernel keeps no historical per-tier statistics.

Runtime state, /sys/kernel/mm/lru_gen/:

$ cat /sys/kernel/mm/lru_gen/enabled
0x0007
$ cat /sys/kernel/mm/lru_gen/min_ttl_ms
0

0x0007 is all three capability bits — LRU_GEN_CORE | LRU_GEN_MM_WALK | LRU_GEN_NONLEAF_YOUNG — reported back after the architecture predicates, so MGLRU is running with both accessed-bit batching optimizations active. min_ttl_ms at 0 means working-set protection is disabled and the timestamps[] field is being used only for the debugfs age_in_ms column, not to veto eviction. Multi-Generational LRU decodes these two files as an ABI in detail.

The debugfs histogram could not be read on this host

/sys/kernel/debug is mode 0700, root-owned, and no passwordless sudo was available to the account running this write-up, so /sys/kernel/debug/lru_gen and lru_gen_full were not read. Every statement about their output format above comes from lru_gen_seq_show() / lru_gen_seq_show_full() in the v6.12 source, not from observed output. Note also that with CONFIG_LRU_GEN_STATS unset this kernel would print xxx for most tier rows anyway. To resolve: sudo cat /sys/kernel/debug/lru_gen on a host where that is permitted. uncertain

Forcing generations to turn over

What can be done without root is to put a process under a memory limit and watch the reclaim counters move. Using systemd’s delegated user cgroup (the memory controller is present in cgroup.subtree_control for user@1000.service), a 1,400 MiB anonymous working set was swept three times inside a 512 MiB limit:

$ systemd-run --user --scope -p MemoryMax=512M -p MemorySwapMax=2G ./hog2.sh
cgroup: /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/mglru-demo2.scope
max: 536870912  swap.max: 2147483648
--- memory.stat (reclaim counters) ---
pgsteal_kswapd   0
pgsteal_direct   1305189
pgscan_kswapd    0
pgscan_direct    1847887
pgrefill         152
pgscan           1847887
pgsteal          1305189
pgactivate       273
pgdeactivate     0
--- memory.events ---
max 17863

And the same run’s effect on the system-wide counters, differenced before and after:

/proc/vmstat counterBeforeAfterDeltaReading
pgscan_anon01,686,568+1,686,568MGLRU counts isolated folios here, not folios examined — see below
pgsteal_anon01,305,148+1,305,14877.4% of “scanned” pages were actually reclaimed
pgscan_direct / pgsteal_direct73,943 / 73,8411,760,511 / 1,378,989+1,686,568 / +1,305,148all of it was direct reclaim by the hog itself, inside its own memcg
pgscan_kswapd / pgsteal_kswapd——+0 / +0kswapd never woke: a memcg limit is hit by the allocating task, not by a node watermark
pswpout / pswpin0 / 0588,331 / 359,340+588,331 / +359,340pages pushed to and pulled back from zram
workingset_refault_anon01,076,090+1,076,090every one of these incremented refaulted[hist][ANON][tier] — the controller was being fed a million data points
pgrefill2,450,3332,450,408+75under MGLRU this counts sorted from scan_folios(), not an active-list scan
pgdeactivate00+0—
pgactivate37,43337,651+218

A forced generation turnover, measured on 7.1.13-200.fc44. What it shows: 1.3 million anonymous pages were evicted and a million of them faulted straight back — a textbook thrash, which is exactly what a working set 2.7x its limit should produce. The insight to take: pgdeactivate is exactly zero, before and after, on a machine whose lifetime pgsteal_file is 4.68 million. That is the “demotion is free” claim from the mental model, measured. PGDEACTIVATE is incremented in only two places in the v6.12 tree — shrink_active_list() in mm/vmscan.c:2117 and the explicit folio_deactivate() / lru_deactivate_file() paths in mm/swap.c — and MGLRU reaches neither. A folio ages by max_seq incrementing around it, and no counter anywhere records that, because no work was done.

Two further details fall out of the same source, and both matter for anyone reading these counters:

  • pgscan_* means something different under MGLRU. The classic shrink_inactive_list() adds nr_scanned — every folio it looked at. MGLRU’s scan_folios() adds isolated, the number it actually pulled off the list (__count_vm_events(item, isolated), mm/vmscan.c:4445-4452). A steal-to-scan ratio near 1.0 is therefore normal under MGLRU and would be extraordinary under the classic LRU. On this host, the lifetime file figures are pgsteal_file 4,680,053 against pgscan_file 4,726,933 — a ratio of 0.990, which under the old accounting would mean a near-perfect oracle and under the new accounting means only “almost everything isolated got reclaimed.”
  • pgrefill is repurposed, not retired. MGLRU does increment it — with sorted, the count of folios sort_folio() handled without reclaiming them (mm/vmscan.c:4448). So a rising pgrefill under MGLRU means “lots of folios are being protected, deferred, or lazily re-sorted”, not “the active list is being scanned”. The +75 above says almost nothing was protected during the thrash, which is correct: a uniformly-swept anonymous working set has no tier structure to protect.

This is the metric divergence that Kalesh Singh and Kairui Song raised at the 2026 LSFMM+BPF planning discussion — that MGLRU’s metrics “differ significantly between the two LRU implementations, and that makes life difficult for components like the Android user-space out-of-memory daemon” (Corbet, LWN, 2026-03-05). The table above is that complaint, reproduced on a laptop.


Failure Modes and Subtleties

Transiently negative nr_pages. Because aging batches its accounting and applies it in reset_batch_size(), the per-generation nr_pages counters are only “eventually consistent” and can read negative momentarily. Code (and humans) reading them must not assume non-negativity at every instant.

Window starvation under no aging. If nothing triggers aging (e.g. an architecture without a hardware young bit and a workload that defeats rmap look-around), the window can fail to produce fresh young generations, degrading MGLRU’s recency resolution. This is why CONFIG_LRU_GEN_WALKS_MMU and the accessed-bit batching switches (0x0002/0x0004 in Multi-Generational LRU) matter for the page-table-walk path.

Tier inflation is bounded. Because a page’s tier is order_base_2(N), frequency growth is logarithmic — a page read a million times is tier ~20 in principle but the array caps at MAX_NR_TIERS = 4, so the highest tier is a catch-all. The feedback loop only needs to distinguish “single-use” (tier 0) from “re-used” (tiers 1+), so four tiers suffice.

Confusing the two “generations.” The single biggest reading hazard: page generations (MAX_NR_GENS = 4, recency of pages within an lruvec) versus memcg generations (MEMCG_NR_GENS = 3, recency of whole cgroups for global reclaim). They are separate mechanisms that share a name.

Anonymous pages can get stuck in the two youngest generations. This is the most-reported real-world failure of the generation model, and it is a known open problem, not a misconfiguration. Zicheng Wang, reporting from an Android OEM, said that with MGLRU “anonymous pages tend to stay within the youngest two generations, causing them to never be reclaimed (and file-backed pages to be reclaimed overly aggressively). Adjusting the swappiness knob does not fix the problem” (Corbet, LWN, 2026-03-05). Kairui Song reported anonymous-reclaim problems independently. Two mechanisms in this note contribute: the fault path inserts folios at max_seq, and — per the controller section above — swappiness under MGLRU is a weight on the refault comparison, not a scan ratio, so turning it up does not force anonymous scanning the way it does in the classic scanner. Honor’s production workaround is to force anon reclaim out of non-foreground cgroups with memory.reclaim; there is no mainline fix as of the 2026 discussion.

Readahead pages land in the youngest generation. Barry Song’s complaint at the same discussion: “when the system performs readahead … it places all of the resulting pages into the youngest generation, even though there is no guarantee that those pages will ever be used at all. That may cause pages actually in use to be reclaimed while leaving the readahead pages in RAM. The traditional LRU, instead, puts those pages onto the inactive list, where they will be reclaimed relatively quickly if they are not referenced again” (LWN 1060967). Note the interaction with the tier table above: speculatively-read pages that are never touched sit at N=0, hence tier 0, so the tier machinery classifies them correctly — but the generation machinery still put them at the hot end of the window, where they will survive several max_seq increments before they are even considered. Tiers defend against streaming reads; generations, on this path, do not. See Readahead.

The generation count cannot be raised. MAX_NR_GENS = 4 is not tunable because the generation number lives in folio->flags, and page flags are the kernel’s most contested scarce resource. MGLRU currently spends three of them; Kairui Song’s proposal at LSFMM+BPF 2026 is to move the generation number elsewhere, which “would make systems with up to 63 generations possible while freeing the three page flags currently used by MGLRU” (LWN 1060967). Until something like that lands, any argument of the form “MGLRU would be better with more generations” is blocked on page-flag budget, not on the algorithm.

Aging is deferred at DEF_PRIORITY, so the window can be stale. As traced in the aging section, get_nr_to_scan() returns early without aging whenever sc->priority == DEF_PRIORITY. On a machine under mild, steady pressure — kswapd trickling along at default priority — the generations can go a long time without turning over, and the debugfs age_in_ms column will show it. This is by design (“try to get away with not aging at the default priority”) but it means a working-set estimate read out of lru_gen on an idle-ish machine may be describing a window that last moved minutes ago. The + verb with force_scan=1 exists precisely to refresh it on demand.


See Also


Sources

All fetched during this write-up. Kernel source and documentation are pinned to the tag named in the URL.

Primary — Linux v6.12 (the vault’s pin):

  • include/linux/mmzone.h — MIN_NR_GENS, MAX_NR_GENS, MAX_NR_TIERS, NR_HIST_GENS, MIN_LRU_BATCH, MAX_LRU_BATCH, MEMCG_NR_GENS, MEMCG_NR_BINS, struct lru_gen_folio, struct lru_gen_mm_state, enum { ... NR_MM_STATS }, and the long design comments quoted throughout.
  • include/linux/mm_inline.h — lru_gen_from_seq(), lru_hist_from_seq(), lru_tier_from_refs(), folio_lru_refs(), folio_lru_gen(), lru_gen_is_active(), lru_gen_update_size().
  • include/linux/page-flags-layout.h — how LRU_REFS_WIDTH is squeezed out of whatever page->flags has left.
  • mm/vmscan.c — should_run_aging(), get_nr_to_scan(), inc_max_seq(), inc_min_seq(), try_to_inc_min_seq(), sort_folio(), isolate_folio(), scan_folios(), isolate_folios(), evict_folios(), read_ctrl_pos(), reset_ctrl_pos(), positive_ctrl_err(), get_tier_idx(), get_type_to_scan(), lru_gen_seq_show(), lru_gen_seq_show_full(), run_aging(), run_eviction().
  • mm/swap.c — the other two PGDEACTIVATE call sites, needed to establish that MGLRU reaches none of them.
  • mm/Kconfig — the # multi-gen LRU { block.
  • Documentation/mm/multigen_lru.rst — the design document.
  • Documentation/admin-guide/mm/multigen_lru.rst — the ABI and the debugfs grammar.

Existence-checking and version pinning (fetched at each tag and compared byte-for-byte):

Secondary — LWN (dated, authored reporting):

Measured on the host (7.1.13-200.fc44, x86-64, 125 GiB RAM, 8 GiB zram swap, 2026-09-11): /boot/config-$(uname -r), /sys/kernel/mm/lru_gen/enabled, /sys/kernel/mm/lru_gen/min_ttl_ms, /proc/vmstat before and after a forced reclaim, and the memory.stat / memory.events of a systemd-run --user --scope -p MemoryMax=512M cgroup.

Not cited, deliberately

lore.kernel.org returns HTTP 403 to this network (re-verified 2026-09-11), so the MGLRU patch-series cover letters — the primary source for every benchmark percentage usually attributed to MGLRU — could not be read. No performance figures are quoted anywhere in this note. Every number here is either a compile-time constant read from source or a measurement taken on the host named above.