The Page Fault Handler

A page fault is the hardware exception the CPU raises when a running instruction touches a virtual address whose page-table entry (PTE) does not currently permit that access — the entry is absent, or present but with the wrong permissions. On Linux the fault is not an error by default: page tables start mostly empty, and the fault is the kernel’s signal to lazily fill them — the mechanism behind demand paging, copy-on-write, swap-in, and automatic NUMA balancing. The handler’s job is to look up the virtual memory area (VMA) covering the faulting address, check whether the attempted access is permitted by that VMA, and if so install or fix the PTE so the instruction can be retried; if not, it delivers SIGSEGV/SIGBUS to a user process or, for a kernel access, either repairs it via the exception table or oopses. This note traces the x86-64 path as it stands in Linux 6.12, this vault’s pinned tag, from the CPU trap through do_user_addr_fault → handle_mm_fault → handle_pte_fault to the per-handler dispatch and the VM_FAULT_* result codes, per the primary source arch/x86/mm/fault.c and mm/memory.c.

What this note owns, and what its siblings own

This note owns the code path a single fault takes: the hardware trap, the entry stub, the error code, the two locking regimes, the permission gate, the page-table descent, the five-way dispatch, the VM_FAULT_* return protocol, the retry machinery, and the kernel-fault/exception-table path. It deliberately stops at the edge of each dispatch arm.

• Demand Paging owns the policy — why page tables start empty at all, the three population flavours (demand-zero, demand-fill, swap-in) in depth, fault-around and how much to populate per fault, and the pre-faulting knobs that override laziness. The boundary readers trip over: “what happens between the trap and the PTE being written” is here; “why is nothing allocated until I touch it” is there.
• Minor and Major Faults owns the classification and accounting — maj_flt/min_flt, PGFAULT/PGMAJFAULT, /proc/<pid>/stat, getrusage(2), and the rule that a retried fault counts as major.
• Copy-on-Write and fork owns dispatch arm (D), do_wp_page. Anonymous vs File-Backed Memory owns the family split that arm (A) makes. The Page Cache owns what filemap_fault reads from. Linux Swap Subsystem and The Swap Cache own arm (B). Automatic NUMA Balancing owns arm (C). Transparent Huge Pages owns the create_huge_pmd branch.

Everything below handle_mm_fault is architecture-independent; everything above it in this note is x86-64, pinned to v6.12 with file-and-line citations throughout.

Version context, as of 2026-09-11 (per kernel.org’s releases.json, the authoritative machine-readable list): mainline is 7.3-rc2, latest stable is 7.2.5, and the longterm lines are 6.18.51, 6.12.109, 6.6.156, 6.1.187, 5.15.220 and 5.10.269. v6.12 is this vault’s pin, not the only longterm kernel — 6.18 is longterm too. Where the fault path differs at a later tag, this note says so and names the tag, and the version table below records how each such claim was measured.

---

Mental Model — Two Locking Regimes, One Dispatch

flowchart TB
  CPU["CPU MMU raises a page fault<br/>(CR2 = faulting addr, error code)"] --> HPF["handle_page_fault()<br/>kernel-space addr? user-space addr?"]
  HPF -->|"kernel addr"| KERN["do_kern_addr_fault()<br/>vmalloc sync / spurious / oops"]
  HPF -->|"user addr"| DUAF["do_user_addr_fault()"]
  DUAF -->|"FAST PATH — only if<br/>FAULT_FLAG_USER and<br/>CONFIG_PER_VMA_LOCK"| RCU["lock_vma_under_rcu()<br/>per-VMA read lock, no mmap_lock"]
  DUAF -->|"slow / fallback"| MMAP["lock_mm_and_find_vma()<br/>mmap_lock (read)"]
  RCU -->|"miss / VM_FAULT_RETRY"| MMAP
  RCU --> ACC["access_error()<br/>permission check vs VMA"]
  MMAP --> ACC
  ACC -->|"denied"| BAD["bad_area_access_error<br/>→ SIGSEGV"]
  ACC -->|"allowed"| HMF["handle_mm_fault()<br/>→ __handle_mm_fault → handle_pte_fault"]
  HMF --> DISP{"dispatch on PTE state"}
  DISP -->|"pte_none, anon VMA"| ANON["do_anonymous_page<br/>(zero page / new page)"]
  DISP -->|"pte_none, file VMA"| FILE["do_fault → filemap_fault<br/>(page cache / readahead I/O)"]
  DISP -->|"!pte_present"| SWAP["do_swap_page<br/>(swap-in: MAJOR fault)"]
  DISP -->|"protnone"| NUMA["do_numa_page<br/>(NUMA balancing)"]
  DISP -->|"write to RO present"| WP["do_wp_page<br/>(COW break)"]
  ANON & FILE & SWAP & NUMA & WP --> RES["VM_FAULT_* result"]
  RES -->|"OK"| RETRY["retry faulting instruction"]
  RES -->|"VM_FAULT_OOM/SIGBUS/SIGSEGV"| SIG["signal or oops"]

The x86-64 user page-fault path in Linux 6.12. What it shows: the hardware trap funnels into handle_page_fault, which splits kernel-space from user-space addresses; the user path first tries the lock-free per-VMA fast path (lock_vma_under_rcu) and falls back to the mmap_lock slow path (lock_mm_and_find_vma) on any miss or retry; both converge on access_error (the permission gate) and then handle_mm_fault, which descends the page-table levels and dispatches in handle_pte_fault to exactly one of the leaf handlers based on the PTE’s state. The insight to take: the fault path is a single funnel with two interchangeable locking front-ends and a five-way dispatch at the bottom; almost everything mm/ does to “fill the illusion” hangs off that dispatch.

---

From the CPU trap to do_user_addr_fault

The entry stub: exc_page_fault

When the memory-management unit (MMU) cannot translate a virtual address — the relevant PTE is not present, or is present but forbids the attempted read/write/execute — it raises the architecture’s page-fault exception. On x86 that is interrupt vector 14, defined as #define X86_TRAP_PF 14 (arch/x86/include/asm/trapnr.h:33). The hardware deposits the faulting linear address in control register CR2 and pushes a 32-bit error code describing the access.

The kernel-side entry point is exc_page_fault, declared with DEFINE_IDTENTRY_RAW_ERRORCODE at arch/x86/mm/fault.c:1493. “RAW” means the macro does not generate the usual entry/exit bookkeeping; exc_page_fault does it by hand, because a page fault is unusual among exceptions in that it may need to sleep — and sleeping requires the entry code to be in exactly the right state first. Four things happen before any mm code runs, in this order:

1. Read the faulting address. address = cpu_feature_enabled(X86_FEATURE_FRED) ? fred_event_data(regs) : read_cr2();. On classic x86-64 the address comes from CR2; on a CPU with FRED (Flexible Return and Event Delivery, Intel’s replacement for the IDT-based event delivery path) it arrives in the event-data field of the pushed frame instead. Reading CR2 early matters: CR2 is a single architectural register, so a nested fault would overwrite it.
2. prefetchw(&current->mm->mmap_lock) — a bare cache-line prefetch-for-write of the address-space lock. It is a pure performance hint, and it is a good indication of where the hot path expects to end up.
3. KVM asynchronous page fault. if (kvm_handle_async_pf(regs, (u32)address)) return;. When Linux runs as a guest, the host reuses vector 14 to tell the guest “the memory you touched is valid from your point of view, but I have it swapped out — go schedule something else and I will interrupt you when it is ready.” That is not a real page fault at all, so it is peeled off before anything else.
4. irqentry_enter(regs) — establishes RCU/context tracking. The source comment is explicit that this is deliberately not ct_irq_enter(), “because a kernel fault on a user space address might sleep.”

Only then does it call handle_page_fault(regs, error_code, address) (arch/x86/mm/fault.c:1469).

sequenceDiagram
    autonumber
    participant HW as CPU / MMU
    participant E as exc_page_fault<br/>(fault.c:1493)
    participant T as trace_page_fault_entries<br/>(fault.c:1456)
    participant H as handle_page_fault<br/>(fault.c:1469)
    participant U as do_user_addr_fault<br/>(fault.c:1211)
    participant K as do_kern_addr_fault<br/>(fault.c:1138)

    HW->>HW: translation fails
    HW->>HW: CR2 := faulting linear address
    HW->>E: vector 14 (page fault) + 32-bit error code
    E->>E: address = read_cr2() or fred_event_data(regs)
    E->>E: prefetchw(&current->mm->mmap_lock)
    E->>E: kvm_handle_async_pf() — host "page not present" event?
    Note over E: if yes: return, this is not a real fault
    E->>E: irqentry_enter(regs) — RCU / context tracking
    E->>H: handle_page_fault(regs, error_code, address)
    H->>T: trace_page_fault_user / _kernel (if tracing on)
    H->>H: kmmio_fault() — mmiotrace probe?
    H->>H: fault_in_kernel_space(address)?
    alt address >= TASK_SIZE_MAX (and not vsyscall)
        H->>K: kernel-space address
    else user-space address
        H->>U: user-space address
        U-->>H: returns (may have re-enabled IRQs)
        H->>H: local_irq_disable()
    end

The x86-64 fault entry chain in Linux 6.12, from the hardware trap to the two address-space handlers. What it shows: four pieces of work — capture the address, prefetch the lock, filter out KVM’s hijacked vector, and fix up RCU state — stand between the CPU exception and any memory-management code, and the user/kernel split happens on the address, not on the privilege level of the faulting code. The insight to take: the entry stub is “raw” precisely because a page fault is the one exception that is allowed to block, so the normal exception prologue (which assumes atomicity) cannot be used; note also step 14 — do_user_addr_fault re-enables interrupts on the way in and the caller has to put them back, a wart the source comments apologise for.

The error code is the whole input

Everything the handler decides flows from the 32-bit error code the hardware pushed, together with the faulting address. The bit assignments are enumerated in arch/x86/include/asm/trap_pf.h:20-30:

packet-beta
0: "PROT"
1: "WR"
2: "USR"
3: "RSVD"
4: "INSTR"
5: "PK"
6: "SHSTK"
7-14: "(reserved)"
15: "SGX"
16-30: "(reserved)"
31: "RMP"

The x86 page-fault error code, bit by bit, transcribed from enum x86_pf_error_code in trap_pf.h at v6.12. What it shows: only nine of the thirty-two bits carry meaning, and they are not a code but a set of independent predicates that the handler tests one at a time. The insight to take: bit 0 is the single most consequential bit in the whole fault path — it separates “there is nothing here” (0) from “there is something here but you may not do that to it” (1), which is exactly the not-present-vs-protection distinction that decides between filling a page and breaking copy-on-write.

Bit	Constant	Set when	Consequence in the handler
0	X86_PF_PROT	The page was present; this was a protection violation, not a missing translation	A write with this bit in a writable VMA is the normal COW break. A read with this bit set is an immediate access_error — a present, readable page must not fault on read
1	X86_PF_WRITE	The access was a write (rather than a read)	Becomes FAULT_FLAG_WRITE; gates VM_WRITE in access_error and read-vs-write in do_anonymous_page
2	X86_PF_USER	The CPU was at CPL 3 (user mode) when it faulted	Deliberately not used to set FAULT_FLAG_USER (see below); used for the SMAP check and the “kernel executing user memory” check
3	X86_PF_RSVD	A reserved bit was set in some page-table entry on the walk	Always a corrupt page table → pgtable_bad() (fault.c:593)
4	X86_PF_INSTR	The access was an instruction fetch	Becomes FAULT_FLAG_INSTRUCTION; enforces no-execute
5	X86_PF_PK	A protection key (PKU) denied the access	Unconditional access_error — filling a page cannot fix it
6	X86_PF_SHSTK	A shadow-stack access (CET)	Forced to FAULT_FLAG_WRITE, and required to land in a VM_SHADOW_STACK VMA
15	X86_PF_SGX	The SGX MMU rejected the access (typically enclave memory destroyed by suspend/resume)	Unconditional access_error, so userspace gets a signal and can rebuild the enclave
31	X86_PF_RMP	An AMD SEV-SNP Reverse Map Table violation	Present in the enum at v6.12; handled in the RMP fault path, not in the ordinary VMA walk

Table: the x86 page-fault error-code bits and what each one causes. The pattern worth noticing is that bits 5, 6, 15 and 31 are all “hardware says no, and no amount of page-filling will change that” — they short-circuit to a signal, while bits 0, 1 and 4 are inputs to a decision the kernel may still resolve successfully.

Note the deliberate asymmetry around bit 2. The comment above do_user_addr_fault is emphatic: “Nothing in here should check X86_PF_USER without a specific justification.” FAULT_FLAG_USER is set from user_mode(regs) — the privilege level of the interrupted register state — not from the error-code bit, so that a copy_to_user() performed by the kernel on a user process’s behalf is still treated as a user access for memory-cgroup OOM purposes, while the SMAP check (which is about who executed the access) correctly uses the hardware bit.

The kernel-vs-user split, and the vsyscall hole in it

handle_page_fault branches on fault_in_kernel_space(address) (fault.c:1119). The predicate is one line — return address >= TASK_SIZE_MAX; — with exactly one carve-out written above it:

bool fault_in_kernel_space(unsigned long address)
{
    /*
     * On 64-bit systems, the vsyscall page is at an address above
     * TASK_SIZE_MAX, but is not considered part of the kernel
     * address space.
     */
    if (IS_ENABLED(CONFIG_X86_64) && is_vsyscall_vaddr(address))
        return false;
 
    return address >= TASK_SIZE_MAX;
}

The legacy vsyscall page lives at VSYSCALL_ADDR, defined as (-10UL << 20) = 0xffffffffff600000 (arch/x86/include/uapi/asm/vsyscall.h:11; the x86-64 memory map documents it as “legacy vsyscall ABI, 4 kB” at -10 MB). It is numerically inside the kernel half, but it is user-reachable by design, so it is routed to the user handler and then emulated before any VMA lookup happens (emulate_vsyscall, fault.c:1320) — the source comment notes that “the vsyscall page does not have a ‘real’ VMA,” which is precisely why the emulation must come first.

On the kernel side (do_kern_addr_fault, fault.c:1138), the legitimate outcomes are narrow: on x86-32 only, a stale vmalloc mapping that needs syncing from init_mm.pgd into this mm’s page tables (vmalloc_fault; the source states the race “does not exist” on x86-64, so the call is inside #ifdef CONFIG_X86_32); the Pentium F00F erratum workaround; and a spurious fault, where a stale TLB entry made the CPU fault on a PTE that is actually fine — spurious_kernel_fault (fault.c:984) re-walks the real page table and silently returns if the hardware was simply behind. Everything else falls through to bad_area_nosemaphore and, absent an exception-table entry, an oops. Note the asymmetry worth remembering: the kernel-address path never takes mmap_lock and never calls handle_mm_fault. Demand paging does not exist for kernel addresses on x86-64.

Early rejects before any VMA lookup

do_user_addr_fault (fault.c:1211) performs a sequence of cheap checks before touching the address space, because each represents a fault that can never be resolved by filling a page. They are, in source order:

1. Kernel executing from user memory (fault.c:1224) — the test is (error_code & (X86_PF_USER | X86_PF_INSTR)) == X86_PF_INSTR: an instruction fetch (bit 4 set) that did not come from user mode (bit 2 clear). That is the kernel trying to execute a user page — the classic ret2usr exploit primitive that SMEP exists to stop. The handler does not even look up the VMA; it goes straight to page_fault_oops after ruling out AMD erratum 93 (which can corrupt RIP so that a legitimate kernel address looks like a user one).
2. kprobes — kprobe_page_fault() is given a chance to claim the fault, wrapped in WARN_ON_ONCE because it should never claim a spurious one.
3. Reserved-bit set (fault.c:1247) — X86_PF_RSVD means a reserved bit was set somewhere on the page-table walk. User page tables must never have one, so this is pgtable_bad().
4. SMAP violation (fault.c:1257) — if Supervisor Mode Access Prevention is enabled, the access did not come from user mode, and EFLAGS.AC is clear (i.e. we are not inside a user_access_begin() / copy_*_user() window), the kernel touched user memory outside a sanctioned window. The comment is blunt: “No extable entry here. This was a kernel access to an invalid pointer.” → page_fault_oops.
5. Fault in atomic or interrupt context (fault.c:1272) — faulthandler_disabled() || !mm. Resolving a fault may sleep, and here we cannot; so bad_area_nosemaphore immediately. The !mm half catches kernel threads, which have no user address space at all.
6. Interrupts were disabled — a WARN_ON_ONCE(!(regs->flags & X86_EFLAGS_IF)), described in the source as a “legacy check - remove this after verifying that it doesn’t trigger.”

Only after all six does the handler call local_irq_enable(), fire the PERF_COUNT_SW_PAGE_FAULTS software event, and translate the hardware error code into the architecture-independent FAULT_FLAG_* vocabulary that mm/ speaks.

flowchart TB
  ENTRY["do_user_addr_fault<br/>(fault.c:1211)"] --> C1{"INSTR fetch<br/>and not USER?"}
  C1 -->|"yes, and not erratum 93"| OOPS1["page_fault_oops<br/>kernel ran user code"]
  C1 -->|"no"| C2{"kprobe claims it?"}
  C2 -->|"yes"| RET["return"]
  C2 -->|"no"| C3{"X86_PF_RSVD set?"}
  C3 -->|"yes"| PGB["pgtable_bad<br/>corrupt page table"]
  C3 -->|"no"| C4{"SMAP on, not USER,<br/>EFLAGS.AC clear?"}
  C4 -->|"yes"| OOPS2["page_fault_oops<br/>bad kernel pointer"]
  C4 -->|"no"| C5{"faulthandler_disabled()<br/>or no mm?"}
  C5 -->|"yes"| BAN["bad_area_nosemaphore<br/>cannot sleep here"]
  C5 -->|"no"| IRQ["local_irq_enable()<br/>perf_sw_event(PAGE_FAULTS)"]
  IRQ --> FLAGS["build FAULT_FLAG_*<br/>from error_code + regs"]
  FLAGS --> VSYS{"is_vsyscall_vaddr?"}
  VSYS -->|"yes, emulated"| RET
  VSYS -->|"no"| LOOKUP["VMA lookup begins"]

The six early rejects inside do_user_addr_fault, in source order, before any address-space structure is consulted. What it shows: every one of these exits is terminal — an oops, a signal, or a return — and none of them can be repaired by allocating memory. The insight to take: the handler is structured as “prove this fault is not resolvable as cheaply as possible, then do the expensive thing,” which is why the four security-relevant checks (SMEP-style execute, reserved bits, SMAP, atomic context) all come before the first lock is taken.

The FAULT_FLAG_* vocabulary

mm/ never sees X86_PF_*. The arch layer translates into enum fault_flag (include/linux/mm_types.h:1383-1398), which every architecture shares:

Flag	Bit	Set by the x86 handler from	Meaning to mm/
FAULT_FLAG_WRITE	1 << 0	X86_PF_WRITE or X86_PF_SHSTK	Treat as a write; gates COW break and dirty-bit setting
FAULT_FLAG_MKWRITE	1 << 1	(not set by the x86 fault path)	Fault was an mkwrite of an existing PTE
FAULT_FLAG_ALLOW_RETRY	1 << 2	FAULT_FLAG_DEFAULT	The handler may drop the lock and return VM_FAULT_RETRY
FAULT_FLAG_RETRY_NOWAIT	1 << 3	FOLL_NOWAIT in faultin_page ([gup.c:1173]https://raw.githubusercontent.com/torvalds/linux/v6.12/mm/gup.c#L1173))	Retry, but do not drop mmap_lock and sleep. Illegal together with FAULT_FLAG_VMA_LOCK — sanitize_fault_flags returns VM_FAULT_SIGSEGV if both are set
FAULT_FLAG_KILLABLE	1 << 4	FAULT_FLAG_DEFAULT	A SIGKILL may abort the fault mid-flight
FAULT_FLAG_TRIED	1 << 5	set on the retry pass	“We have been here before” — suppresses a second async readahead in filemap_fault, and forces the fault to be counted as major
FAULT_FLAG_USER	1 << 6	user_mode(regs) — not X86_PF_USER	Arms memcg OOM handling (mem_cgroup_enter_user_fault) and enables the per-VMA fast path
FAULT_FLAG_REMOTE	1 << 7	FOLL_REMOTE in faultin_page (gup.c:1160)	The fault is for another task’s mm, e.g. process_vm_readv or ptrace
FAULT_FLAG_INSTRUCTION	1 << 8	X86_PF_INSTR	Instruction fetch; feeds arch_vma_access_permitted
FAULT_FLAG_INTERRUPTIBLE	1 << 9	FAULT_FLAG_DEFAULT	Non-fatal signals may also abort the fault
FAULT_FLAG_UNSHARE	1 << 10	faultin_page when GUP must unshare (gup.c:1182) — FOLL_PIN && !FOLL_WRITE on a shared anon page, signalled by -EMLINK	Break COW without being a write, so a long-term pin cannot be silently detached from the process’s view. Illegal together with FAULT_FLAG_WRITE (VM_BUG_ON)
FAULT_FLAG_ORIG_PTE_VALID	1 << 11	set inside handle_pte_fault	vmf->orig_pte has been latched and may be read
FAULT_FLAG_VMA_LOCK	1 << 12	set when entering under the per-VMA lock	Tells every leaf handler “you hold a VMA lock, not mmap_lock” — several bail out with VM_FAULT_RETRY on seeing it

Table: enum fault_flag at v6.12, with the bit values and which ones the x86-64 fault path actually sets. FAULT_FLAG_DEFAULT is (FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE | FAULT_FLAG_INTERRUPTIBLE) (include/linux/mm.h:501-503) — so by default a user fault is allowed to sleep, allowed to retry, and interruptible by any signal. Only three combinations of ALLOW_RETRY/TRIED are legal; the fourth (!ALLOW_RETRY && TRIED) is documented as “illegal and should never be used.”

---

Finding the VMA: the per-VMA fast path vs. the mmap_lock fallback

To resolve a fault the kernel must find the VMA that contains the address — the VMA carries the permissions, the backing (anonymous, file, special), and the vm_ops callbacks. Historically every lookup took the per-mm reader/writer semaphore mmap_lock in read mode, and that lock became one of the kernel’s worst scalability bottlenecks: a multi-threaded process faulting on many cores serializes every page fault on a single rwsem, even though the threads fault in disjoint VMAs (LWN, Concurrent page-fault handling with per-VMA locks).

Why this lock exists at all — the design rationale

The two-regime structure is not an optimisation someone bolted on; it is the current state of a decade-long argument, and knowing the argument explains several otherwise-arbitrary details.

Matthew Wilcox framed the problem at the 2023 LSFMM+BPF summit as two distinct issues, not one: “contention for the per-process mmap_lock lock and priority inversions between monitoring tasks and the real workload (which can come about, for example, when the monitoring task reads data from /proc/pid/smaps)” (LWN, Improving page-fault scalability, Corbet, 29 May 2023). The second half matters as much as the first in production: a low-priority monitoring agent sampling smaps takes mmap_lock and can stall the high-priority thread it is monitoring. Fine-grained locking fixes both.

The same session records the intended end state, which reframes the fast path as a migration rather than a shortcut: work “continues to grow the number of cases that can be handled without resorting to the mmap_lock. Future projects will add more of these cases, including initiating and waiting for I/O, providing the data for /proc/pid/smaps, faults handled via userfaultfd(), faults in VMAs created by device drivers, and so on. Someday there will be no cases left to convert, and mmap_lock can be removed entirely.” Every VM_FAULT_RETRY-under-FAULT_FLAG_VMA_LOCK bail-out you find in mm/ is one of those unconverted cases.

Wilcox also described, in that session, the exact design the code in filemap_fault implements: “take a reference to the file being read, drop the lock, then sleep. Once the I/O completes, the fault-handling process would restart from the beginning to catch up with any other address-space changes that may have taken place.” He asked the room whether that was right or whether the kernel should instead block changes to the faulting VMA for the duration. Michal Hocko’s answer is why swap and file faults still differ today: in the swap case “the owning process could unmap the memory while waiting for the faulting page to be read in,” which holding the VMA lock would prevent — but that “would not work as well for the file-backed case, where the VMAs do not map to process-visible objects.”

The per-VMA fast path (lock_vma_under_rcu)

Linux 6.12 builds with CONFIG_PER_VMA_LOCK, which is def_bool y but gated on ARCH_SUPPORTS_PER_VMA_LOCK && MMU && SMP (mm/Kconfig:1268-1270). The architecture symbol is selected by config X86_64, not by config X86 (arch/x86/Kconfig:30) — so a 32-bit x86 kernel, or any uniprocessor build, gets no per-VMA locking at all and always takes mmap_lock. (LOCK_MM_AND_FIND_VMA, by contrast, is selected by config X86 at line 299, so the slow path is common to both.) So on a normal SMP x86-64 build, the feature is compiled in and on by default. The per-VMA locking work was merged in Linux 6.4 (June 2023) and needed a stabilization round — a fork()-vs-fault data-corruption bug and the StackRot fix forced explicit per-VMA serialization, resolved in 6.4.3 (July 2023) (LWN, Stabilizing per-VMA locking). Per Liam Howlett’s report at the 2024 LSFMM+BPF summit, fault-handling throughput doubled in 6.4 and reached nearly 3× the pre-work baseline by 6.6 (LWN, Faster page faults with RCU-protected VMA walks).

For a user-space fault, do_user_addr_fault calls lock_vma_under_rcu(mm, address) before ever taking mmap_lock. Under the RCU read lock it walks the maple tree (mas_walk) to find the VMA, then takes a lightweight per-VMA read lock. If that succeeds and the address is still inside the VMA’s bounds, the fault is handled with the flag FAULT_FLAG_VMA_LOCK set and no mmap_lock at all. Crucially, disjoint faults on different VMAs no longer contend. If lock_vma_under_rcu returns NULL (VMA not found, or being concurrently modified), or if handle_mm_fault returns VM_FAULT_RETRY under the VMA lock, control falls through the goto lock_mmap to the slow path.

sequenceDiagram
    autonumber
    participant A as do_user_addr_fault
    participant R as lock_vma_under_rcu<br/>(memory.c:6231)
    participant M as lock_mm_and_find_vma<br/>(memory.c:6169)
    participant F as handle_mm_fault

    Note over A: flags = FAULT_FLAG_DEFAULT
    A->>A: FAULT_FLAG_USER set?
    Note over A: if not (kernel touched user memory),<br/>skip straight to lock_mmap
    A->>R: lock_vma_under_rcu(mm, address)
    alt VMA found and per-VMA read lock taken
        R-->>A: locked vma
        A->>A: access_error(error_code, vma)?
        A->>F: handle_mm_fault(vma, addr, flags or FAULT_FLAG_VMA_LOCK)
        alt no VM_FAULT_RETRY
            F-->>A: done — count VMA_LOCK_SUCCESS, goto done
        else VM_FAULT_RETRY (lock already dropped by callee)
            F-->>A: VM_FAULT_RETRY
            A->>A: count VMA_LOCK_RETRY
            A->>A: if VM_FAULT_MAJOR, set FAULT_FLAG_TRIED
            A->>A: fault_signal_pending()? bail out
        end
    else miss / detached / out of bounds
        R-->>A: NULL — count VMA_LOCK_ABORT
    end
    A->>M: lock_mm_and_find_vma(mm, address, regs)
    Note over M: get_mmap_lock_carefully -> mmap_read_trylock,<br/>then extend a VM_GROWSDOWN stack VMA if needed
    M-->>A: vma, with mmap_lock held for read
    A->>A: access_error(error_code, vma)?
    A->>F: handle_mm_fault(vma, addr, flags)
    alt VM_FAULT_RETRY
        F-->>A: mmap_lock already released
        A->>A: set FAULT_FLAG_TRIED, goto retry
    else resolved or error
        F-->>A: result
        A->>A: mmap_read_unlock(mm)
    end

The two-regime locking dance in do_user_addr_fault at v6.12. What it shows: the per-VMA fast path is attempted first and exactly once — on any failure, and on a VM_FAULT_RETRY under the VMA lock, control falls through to the mmap_lock path, which itself loops on VM_FAULT_RETRY. The insight to take: the fast path has no retry loop of its own; it is strictly a “try once, then downgrade” optimisation, and the mmap_lock path remains the correctness backstop that every complicated case (stack growth, userfaultfd, device-private swap entries) ends up in. Note step 2: a fault without FAULT_FLAG_USER — the kernel touching user memory through copy_to_user — skips the fast path entirely.

What changed across releases, established by existence-checking

Version claims about per-VMA locking are easy to get wrong because the subsystem is still moving. The following were established by fetching the same paths at successive tags and recording what was there — not from release notes:

Claim	How it was checked	Result
lock_vma_under_rcu did not exist before 6.4	grep the symbol in mm/memory.c at v6.3 and v6.4	0 hits at v6.3, present at v6.4 — the feature landed in 6.4
It lives in mm/memory.c in 6.12	grep at v6.12	Present, memory.c:6231
It moved to mm/mmap_lock.c after 6.12	grep both files at v6.18	0 hits in mm/memory.c, 2 in mm/mmap_lock.c
The 6.12 lock is an rwsem; the 6.18 lock is a refcount	Read vma_start_read at each tag	v6.12: down_read_trylock(&vma->vm_lock->lock) (include/linux/mm.h:692-722). v6.18: __refcount_inc_not_zero_limited_acquire(&vma->vm_refcnt, …) with VMA_LOCK_OFFSET/VMA_REF_LIMIT. By v7.2 the constant is renamed VM_REFCNT_EXCLUDE_READERS_FLAG
Per-VMA locking is “available in all builds”	Read config PER_VMA_LOCK at v6.12, v6.18, v7.0, v7.1, v7.2, v7.3-rc2	False at every one of those tags. depends on ARCH_SUPPORTS_PER_VMA_LOCK && MMU && SMP is still there at v7.3-rc2. The “mm: Make per-VMA locks available in all builds” series is a patch posting by Dave Hansen dated 29 April 2026 whose own cover letter opens “I hope I’m not missing something big here” — it is a proposal, not merged code. Do not cite it as landed
The handle_pte_fault dispatch shape changed	Read the dispatch at v6.12 and v7.2	Unchanged. do_pte_missing / do_swap_page / do_numa_page / do_wp_page, in that order, at v7.2 memory.c:6379-6395 exactly as at v6.12

Table: version claims about the fault path, each one measured rather than inferred. Checked 2026-09-11, when mainline was 7.3-rc2 and the newest stable release was 7.2.5 (per kernel.org/releases.json). The last row is the reassuring one: the structure described in this note — two locking front-ends, one dispatch — has been stable from 6.12 through 7.2 even though the internals of the fast path were rebuilt twice.

The externally visible behaviour is identical across all three implementations — try a per-VMA lock, fall back to mmap_lock, set FAULT_FLAG_VMA_LOCK so the leaf handlers know which lock they hold — so everything this note says about the shape of the path holds; only the lock’s internals differ. The code excerpts below are from 6.12.

The 6.12 lock_vma_under_rcu body, lightly annotated:

struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
                                          unsigned long address)
{
    MA_STATE(mas, &mm->mm_mt, address, address); /* maple-tree walk state */
    struct vm_area_struct *vma;
 
    rcu_read_lock();                 /* (1) RCU protects the VMA from being freed */
retry:
    vma = mas_walk(&mas);            /* (2) find VMA covering 'address' */
    if (!vma)
        goto inval;                  /*     no VMA → fall back to mmap_lock path */
 
    if (!vma_start_read(vma))        /* (3) try the per-VMA read lock */
        goto inval;
 
    if (vma->detached) {             /* (4) VMA was isolated after we found it */
        vma_end_read(vma);
        count_vm_vma_lock_event(VMA_LOCK_MISS);
        goto retry;                  /*     replaced by another VMA — retry walk */
    }
    /* address might have moved out of [vm_start, vm_end) before we locked: */
    if (unlikely(address < vma->vm_start || address >= vma->vm_end))
        goto inval_end_read;
 
    rcu_read_unlock();
    return vma;                      /* (5) stable, locked VMA — proceed lock-free */
inval_end_read:
    vma_end_read(vma);
inval:
    rcu_read_unlock();
    count_vm_vma_lock_event(VMA_LOCK_ABORT);
    return NULL;                     /* caller falls back to mmap_lock */
}

Line (3) is the heart: vma_start_read succeeds only if no writer holds the VMA. Any structural change to the address space (mmap, munmap, mprotect, mremap) write-locks the affected VMAs, which makes concurrent readers fail fast and fall back rather than observe a half-modified VMA. This is not lockless speculation — the VMA is genuinely locked, just at finer grain than the whole mm (LWN, Faster page faults with RCU-protected VMA walks).

That distinction is the punchline of a fourteen-year story. The alternative approach, speculative page-fault handling, tried to resolve faults with no VMA lock at all: “do as much work as possible that doesn’t depend on the state of the VMA, then check to see if anything has changed before changing the process’s address space.” Those patches “first appeared in 2009 and have been discussed and improved upon intermittently by various kernel developers through the years, but this work has not found its way into the kernel”; Laurent Dufour’s 2017 revival reported “a 20% speed improvement loading a 2TB database” and still did not merge (LWN, Another attempt at speculative page-fault handling, Nur Hussein, 14 August 2017). What eventually shipped in 6.4 was the conservative version of the same idea: keep a real lock, but make it per-VMA, and fall back whenever anything is complicated. The goto inval paths in the function above are that fallback, and they are why the design was mergeable when speculation was not.

The mmap_lock slow path (lock_mm_and_find_vma)

When the fast path bails, do_user_addr_fault enters the lock_mmap/retry block and calls lock_mm_and_find_vma(mm, address, regs) (mm/memory.c:6169). Three things make it more than “take the lock and look up the VMA”:

It takes the lock carefully. get_mmap_lock_carefully tries mmap_read_trylock first. If that fails and the fault came from kernel mode, it consults search_exception_tables(exception_ip(regs)) before blocking. The reason is a genuine deadlock: if a kernel bug faults while the same thread already holds mmap_lock for writing, blocking on the read side would hang forever. Only an instruction that is registered as allowed to fault (see the exception-table section below) is permitted to wait.

It grows the stack. A fault a little below a VM_GROWSDOWN VMA’s vm_start is not an error — it is the process’s stack expanding. But expansion is a structural change, which needs the write lock. The code first attempts mmap_upgrade_trylock(), which in v6.12 is a stub that always returns false (the source says outright: “We don’t have this operation yet”), so in practice every stack-growth fault takes the second path: drop the read lock, re-acquire for writing (again checking exception tables for a kernel-mode fault), re-look-up the VMA, expand_stack_locked(), and finally mmap_write_downgrade() back to a read lock before returning. That is why stack growth can never be served by the per-VMA fast path — the fast path holds a read lock on one VMA and has no way to promote it.

It cleans up on failure. If no VMA is found and the address cannot be a legal stack expansion, lock_mm_and_find_vma releases the lock and returns NULL; the caller then goes to bad_area_nosemaphore → SIGSEGV without any further unlocking. The two exits are asymmetric on purpose, and getting that wrong is a classic source of lock leaks in architecture ports.

---

The permission gate: access_error

Whichever lock was taken, the next step is access_error(error_code, vma) (arch/x86/mm/fault.c:1053). This is the permission gate, and it is worth being precise about what question it answers. It does not ask “is the page there?” — it never looks at a PTE. It asks: given the VMA’s flags, is this kind of access allowed here at all? A “no” is unfixable and becomes a signal; a “yes” merely means the fault is worth trying to resolve.

The checks run in this order, and the order matters because the early ones are unconditional denials:

flowchart TB
  IN["access_error(error_code, vma)"] --> PK{"X86_PF_PK?<br/>protection key denied"}
  PK -->|"yes"| E1["return 1 — unconditional<br/>'can never result in a follow-up<br/>action to resolve the fault, like a COW'"]
  PK -->|"no"| SGX{"X86_PF_SGX?"}
  SGX -->|"yes"| E2["return 1 — enclave memory<br/>destroyed; let userspace rebuild"]
  SGX -->|"no"| AVAP{"arch_vma_access_permitted<br/>(pkey vs VMA)?"}
  AVAP -->|"denied"| E3["return 1 — refuse *before*<br/>filling a page we could not use"]
  AVAP -->|"allowed"| SHS{"X86_PF_SHSTK?"}
  SHS -->|"yes"| SHS2{"VMA has VM_SHADOW_STACK<br/>and VM_WRITE?"}
  SHS2 -->|"no"| E4["return 1"]
  SHS2 -->|"yes"| OK1["return 0"]
  SHS -->|"no"| WR{"X86_PF_WRITE?"}
  WR -->|"yes"| WR2{"VMA is VM_SHADOW_STACK?"}
  WR2 -->|"yes"| E5["return 1 — ordinary write<br/>into a shadow stack"]
  WR2 -->|"no"| WR3{"VMA has VM_WRITE?"}
  WR3 -->|"no"| E6["return 1 — SIGSEGV/SEGV_ACCERR"]
  WR3 -->|"yes"| OK2["return 0 — permitted<br/>even if the PTE is read-only:<br/>that is the COW case"]
  WR -->|"no (read or exec)"| RD{"X86_PF_PROT set?<br/>page was present"}
  RD -->|"yes"| E7["return 1 — a present readable page<br/>must not fault on read"]
  RD -->|"no"| RD2{"vma_is_accessible(vma)?"}
  RD2 -->|"no"| E8["return 1 — PROT_NONE region"]
  RD2 -->|"yes"| OK3["return 0"]

The access_error decision tree at v6.12, transcribed branch for branch. What it shows: eight distinct ways to say “no” and three to say “yes,” with every “no” reachable without consulting a single page-table entry. The insight to take: the single most important branch is the bottom-right one — a write to a writable VMA returns 0 even when the PTE is present and read-only. That is not an oversight; it is the doorway through which every copy-on-write break passes. If access_error treated “present + read-only + write” as an error, fork() would be impossible.

Access	VMA flags	Error-code bits	Verdict	What happens next
Read, page absent	VM_READ	—	allowed	do_pte_missing → demand-zero or demand-fill
Read, page absent	PROT_NONE (no read/exec)	—	denied	SIGSEGV / SEGV_ACCERR
Read, page present	any	PROT	denied	Should be impossible for a legitimate mapping; treated as a pkey/special case
Write, page absent	VM_WRITE	WRITE	allowed	do_pte_missing → allocate a private page
Write, page absent	no VM_WRITE	WRITE	denied	SIGSEGV — e.g. writing to a PROT_READ mapping
Write, page present, PTE read-only	VM_WRITE	PROT + WRITE	allowed	do_wp_page — the COW break
Write, page present, PTE read-only	no VM_WRITE	PROT + WRITE	denied	SIGSEGV — e.g. writing to a mprotect(PROT_READ) region
Instruction fetch	no VM_EXEC	INSTR	denied	SIGSEGV — NX enforcement, via arch_vma_access_permitted
Any	any	PK	denied	SIGSEGV / SEGV_PKUERR
Shadow-stack write	not VM_SHADOW_STACK	SHSTK	denied	SIGSEGV — CET violation

Table: the common (access, VMA, error-code) combinations and access_error’s verdict. Read rows 4 and 6 together: they are the same user-visible operation — “store to an address” — and the whole difference between “allocate a fresh zero page” and “copy someone else’s page” is one hardware bit.

If access_error returns nonzero the handler calls bad_area_access_error (fault.c:872), which for a user task raises SIGSEGV with si_code SEGV_ACCERR, or SEGV_PKUERR plus the offending key in si_pkey when bad_area_access_from_pkeys() says a protection key was responsible; for a kernel-mode access it runs the exception-table fixup instead. Note that the fast path and the slow path call it with different arguments — bad_area_access_error(regs, error_code, address, NULL, vma) under the VMA lock (NULL mm, because there is no mmap_lock to release) versus (…, mm, vma) under mmap_lock. That NULL is the signal that tells the error path which lock to drop.

The pkey case carries a documented, unfixable race that is worth knowing if you ever debug a SEGV_PKUERR. bad_area_access_error reports si_pkey = vma_pkey(vma) — the key currently on the VMA — but the source is explicit that this “does not guarantee that the VMA we find here was the one that we faulted on,” and walks through the interleaving: thread T1 faults on a PTE with pkey=4; before T1 reaches the handler, thread T2 calls mprotect_key(foo, PAGE_SIZE, pkey=5); T1 then reports pkey=5. The signal is correct that a protection key denied the access; the key number it names can be stale.

One subtlety that trips people up: access_error is called before handle_mm_fault, but handle_mm_fault also calls arch_vma_access_permitted again at mm/memory.c:6057, returning VM_FAULT_SIGSEGV if it now fails. The duplication is deliberate — the arch check is about this architecture’s hardware keys, while the generic check also covers remote faults (FAULT_FLAG_REMOTE) arriving through get_user_pages_remote(), which never pass through access_error at all.

---

handle_mm_fault → __handle_mm_fault → handle_pte_fault

handle_mm_fault(vma, address, flags, regs) (mm/memory.c:6042) is the architecture-independent entry into mm/, and the boundary where x86-specific vocabulary stops. Everything below it is portable. It does seven things, in order:

1. __set_current_state(TASK_RUNNING) — the fault may have been reached from a path that had already set a sleeping state; reset it so the scheduler does not lose the task.
2. sanitize_fault_flags(vma, &flags) (memory.c:6001) — three WARN_ON_ONCE invariants, each returning VM_FAULT_SIGSEGV if violated: FAULT_FLAG_UNSHARE together with FAULT_FLAG_WRITE; a write fault on a VMA lacking VM_MAYWRITE; and FAULT_FLAG_VMA_LOCK together with FAULT_FLAG_RETRY_NOWAIT (illegal “because of the assumption that lock is dropped on VM_FAULT_RETRY”). It also quietly clears FAULT_FLAG_UNSHARE on a non-COW mapping, downgrading the request to an ordinary read fault.
3. A second arch_vma_access_permitted check — see the note at the end of the previous section on why this is not redundant.
4. is_droppable = !!(vma->vm_flags & VM_DROPPABLE) — the VMA flag behind mmap’s MAP_DROPPABLE (0x08, documented in include/uapi/linux/mman.h as “Zero memory under memory pressure”). On the way out, handle_mm_fault does ret &= ~VM_FAULT_OOM for such a VMA — the source comment reads “If the mapping is droppable, then errors due to OOM aren’t fatal.” The caller sees a plain failure rather than an OOM kill, because the contents were declared disposable up front.
5. mem_cgroup_enter_user_fault() for FAULT_FLAG_USER — arms the memory-cgroup OOM machinery so a cgroup-limit violation in this fault can be handled by mem_cgroup_oom_synchronize() on the way out, rather than by blowing up inside the allocator.
6. lru_gen_enter_fault(vma) — sets current->in_lru_fault to vma_has_recency(vma), telling MGLRU whether accesses during this fault should count toward page aging. A VMA marked MADV_SEQUENTIAL/MADV_RANDOM has no recency and should not promote pages.
7. Dispatch: is_vm_hugetlb_page(vma) → hugetlb_fault() (mm/hugetlb.c:6294), a completely separate fault handler with its own reservation and locking rules; everything else → __handle_mm_fault.

On the way out it calls mm_account_fault (memory.c:5927), which bumps PGFAULT, decides major-vs-minor, and increments current->maj_flt or current->min_flt — the subject of Minor and Major Faults, which owns those counters and is where the “a retried fault is counted major” rule is explained. Note the source’s own warning immediately after the dispatch: “It is no longer safe to dereference vma-> after this point, because mmap_lock might have been dropped.” That is why mm is cached into a local at function entry.

__handle_mm_fault — descending the page tables, allocating as it goes

__handle_mm_fault (memory.c:5816) builds the struct vm_fault — the mutable scratchpad that is threaded through every leaf handler — and then walks the page-table hierarchy for the faulting address, allocating intermediate levels on demand. struct vm_fault is defined at include/linux/mm.h:547; note that the brief’s common claim that it lives in mm_types.h is wrong — mm_types.h holds enum fault_flag and enum vm_fault_reason, while the struct itself is in mm.h. Its first five fields sit inside an anonymous const struct (vma, gfp_mask, pgoff, address, real_address), so a leaf handler cannot accidentally retarget the fault; address is page-aligned and real_address is the byte the instruction actually touched.

flowchart TB
  START["__handle_mm_fault(vma, address, flags)<br/>build struct vm_fault"] --> PGD["pgd = pgd_offset(mm, address)"]
  PGD --> P4D["p4d = p4d_alloc(mm, pgd, address)"]
  P4D -->|"NULL"| OOM["return VM_FAULT_OOM"]
  P4D --> PUD["vmf.pud = pud_alloc(mm, p4d, address)"]
  PUD -->|"NULL"| OOM
  PUD --> HPUD{"thp_vma_allowable_order<br/>at PUD_ORDER?"}
  HPUD -->|"yes, pud_none"| CHPUD["create_huge_pud<br/>(PUD-order: devmap / DAX)"]
  CHPUD -->|"not FALLBACK"| DONE["return result"]
  HPUD -->|"no / FALLBACK"| PMD["vmf.pmd = pmd_alloc(mm, vmf.pud, address)"]
  CHPUD -->|"FALLBACK"| PMD
  PMD -->|"NULL"| OOM
  PMD --> HPMD{"thp_vma_allowable_order<br/>at PMD_ORDER?"}
  HPMD -->|"yes, pmd_none"| CHPMD["create_huge_pmd<br/>(2 MiB THP)"]
  CHPMD -->|"not FALLBACK"| DONE
  HPMD -->|"no / FALLBACK"| PTE["handle_pte_fault(&vmf)"]
  CHPMD -->|"FALLBACK"| PTE
  HPMD -->|"pmd already huge"| WPH["wp_huge_pmd / huge_pmd_set_accessed"]
  WPH -->|"FALLBACK"| PTE
  WPH -->|"handled"| DONE

The page-table descent inside __handle_mm_fault at v6.12. What it shows: each level is *_alloc, not *_offset — the table below is created if it does not exist, which is exactly why an untouched mmap costs no page-table memory. At both the PUD and PMD levels the walk pauses to ask whether a huge page should be created instead of descending further. The insight to take: VM_FAULT_FALLBACK is the glue of this diagram — a huge-page attempt that fails does not fail the fault, it simply rejoins the small-page path one level down. Every NULL return from an *_alloc becomes VM_FAULT_OOM, which is how memory exhaustion in the page tables themselves surfaces.

Two details the diagram compresses. First, the PUD level has a retry_pud label: after pmd_alloc, the code re-checks pud_trans_unstable(vmf.pud) and jumps back, because a concurrent fault may have collapsed the PUD into a huge entry while we were allocating below it. Second, at the PMD level the walk can find a swap PMD — is_swap_pmd(vmf.orig_pmd) — which in practice means a PMD-sized migration entry; the handler calls pmd_migration_entry_wait() and returns 0, i.e. “no error, just try again,” letting the instruction re-fault once migration completes. Both are examples of a general pattern: the fault path never holds a lock across a wait it does not have to, and prefers to unwind and re-fault.

The five-way dispatch in handle_pte_fault

handle_pte_fault (mm/memory.c:5732) is the decision point — the narrowest place in the whole subsystem, where one pte_t value decides which of five very different mechanisms runs.

Before it can dispatch it must latch the PTE, and the way it does so is itself instructive. If the PMD is empty it does not allocate a PTE table: vmf->pte = NULL and it defers, because — per the source comment — “vm_ops->fault may want to allocate huge page, and if we expose page table for an instant, it will be difficult to retract from concurrent faults and from rmap lookups.” Otherwise it calls pte_offset_map_nolock() (which maps the PTE page and identifies the split page-table lock without taking it), reads the entry with ptep_get_lockless(), and records FAULT_FLAG_ORIG_PTE_VALID so later code knows vmf->orig_pte is meaningful. A pte_none() result is then normalised back to vmf->pte = NULL, folding “no PTE table” and “empty PTE” into one case.

if (!vmf->pte)                                   /* PMD empty, or PTE was none */
    return do_pte_missing(vmf);                  /* (A) first-ever population */
 
if (!pte_present(vmf->orig_pte))                 /* swap / migration / marker */
    return do_swap_page(vmf);                    /* (B) swap-in, often MAJOR */
 
if (pte_protnone(vmf->orig_pte) && vma_is_accessible(vmf->vma))
    return do_numa_page(vmf);                    /* (C) NUMA hinting fault */
 
spin_lock(vmf->ptl);                             /* only now is the ptl taken */
entry = vmf->orig_pte;
if (unlikely(!pte_same(ptep_get(vmf->pte), entry))) {
    update_mmu_tlb(vmf->vma, vmf->address, vmf->pte);
    goto unlock;                                 /* raced: someone else fixed it */
}
if (vmf->flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) {
    if (!pte_write(entry))
        return do_wp_page(vmf);                  /* (D) COW break / unshare */
    else if (likely(vmf->flags & FAULT_FLAG_WRITE))
        entry = pte_mkdirty(entry);              /* (E) already writable */
}
entry = pte_mkyoung(entry);
if (ptep_set_access_flags(vmf->vma, vmf->address, vmf->pte, entry,
                          vmf->flags & FAULT_FLAG_WRITE))
    update_mmu_cache_range(vmf, vmf->vma, vmf->address, vmf->pte, 1);
else if (vmf->flags & FAULT_FLAG_TRIED)
    goto unlock;                                 /* skip flush on a retried fault */
else if (vmf->flags & FAULT_FLAG_WRITE)
    flush_tlb_fix_spurious_fault(vmf->vma, vmf->address, vmf->pte);

Two lines deserve attention even though they resolve nothing. The pte_same() re-check under vmf->ptl is the whole reason the lockless read above is safe: the value was read without the lock, so before acting on it the handler re-reads under the lock and, if another CPU already installed a good PTE, simply unlocks and returns 0 — the instruction is retried and will not fault again. And the tail, flush_tlb_fix_spurious_fault, exists purely for spurious faults: on SMP a stale TLB entry can make one CPU fault on a PTE that is perfectly fine, and the only correct response is to flush the stale entry and return success.

flowchart TB
  IN["handle_pte_fault(vmf)<br/>(memory.c:5732)"] --> LATCH["latch PTE:<br/>pte_offset_map_nolock +<br/>ptep_get_lockless"]
  LATCH --> Q1{"vmf->pte == NULL?<br/>(no PTE table, or pte_none)"}
  Q1 -->|"yes"| A["(A) do_pte_missing<br/>memory.c:3960"]
  A --> A1{"vma_is_anonymous?"}
  A1 -->|"yes"| AZ["do_anonymous_page<br/>read: shared zero page<br/>write: fresh folio"]
  A1 -->|"no"| AF["do_fault<br/>calls vm_ops->fault,<br/>usually filemap_fault"]
  Q1 -->|"no"| Q2{"not pte_present(orig_pte)?"}
  Q2 -->|"yes"| B["(B) do_swap_page<br/>memory.c:4200"]
  B --> B1["swap entry: swap-in<br/>migration entry: wait<br/>hwpoison: VM_FAULT_HWPOISON<br/>pte marker: uffd-wp or POISONED<br/>device-private: migrate_to_ram"]
  Q2 -->|"no"| Q3{"pte_protnone and<br/>vma_is_accessible?"}
  Q3 -->|"yes"| C["(C) do_numa_page<br/>memory.c:5546"]
  Q3 -->|"no"| LOCK["spin_lock(vmf->ptl)<br/>pte_same() re-check"]
  LOCK -->|"changed"| RACE["raced — unlock, return 0"]
  LOCK --> Q4{"WRITE or UNSHARE<br/>and not pte_write?"}
  Q4 -->|"yes"| D["(D) do_wp_page<br/>memory.c:3655"]
  Q4 -->|"no"| E["(E) pte_mkyoung / pte_mkdirty<br/>ptep_set_access_flags,<br/>or flush a spurious TLB entry"]

The five-way dispatch in handle_pte_fault at v6.12, with the sub-dispatch inside each arm. What it shows: the branch order is not arbitrary — it goes from “nothing exists” to “something exists elsewhere” to “something exists here but is deliberately poisoned” to “something exists here and is fine,” and only the last case needs the page-table lock. The insight to take: branch (B) is much wider than “swap.” A non-present PTE encodes at least five different things — a swap slot, an in-flight migration, a hardware-poisoned page, a userfaultfd write-protect marker, and a device-private page — and do_swap_page is the demultiplexer for all of them.

PTE state the fault finds	Test in handle_pte_fault	Handler	Typical cost	Who owns the explanation
No PTE table at all (PMD empty)	pmd_none(*vmf->pmd) then vmf->pte = NULL	do_pte_missing	Minor, plus a page-table allocation	Demand Paging
Empty PTE (pte_none)	normalised to vmf->pte = NULL	do_pte_missing	Minor	Demand Paging
Swap entry	!pte_present	do_swap_page	Major if it must read the device; minor on a swap-cache hit	Linux Swap Subsystem
Migration entry	!pte_present, is_migration_entry	do_swap_page then migration_entry_wait	Blocks until migration finishes, then re-faults	Page Migration
Hardware-poisoned entry	!pte_present, is_hwpoison_entry	do_swap_page returns VM_FAULT_HWPOISON	SIGBUS	Documentation/mm/hwpoison.rst
PTE marker (uffd-wp, POISONED)	!pte_present, is_pte_marker_entry	handle_pte_marker	Varies — may hand the fault to userspace	Documentation/admin-guide/mm/userfaultfd.rst
Device-private entry	!pte_present, is_device_private_entry	migrate_to_ram callback; bails out with VM_FAULT_RETRY under the VMA lock	Device-dependent	Documentation/mm/hmm.rst
Present but PROT_NONE	pte_protnone && vma_is_accessible	do_numa_page	Minor; may trigger a page migration	Automatic NUMA Balancing
Present, read-only, write fault	pte_present, !pte_write, FAULT_FLAG_WRITE	do_wp_page	Minor; one page copy in the worst case	Copy-on-Write and fork
Present and already writable	falls through to case (E)	pte_mkdirty / pte_mkyoung	Cheapest possible — often a spurious fault	this note

Table: every PTE state a fault can encounter at v6.12, and which handler claims it. The column that matters most is the last one: this note owns the dispatch, and each sibling owns one arm of it. If you want to know what do_swap_page does, follow the link; if you want to know why control got there, you are in the right place.

The five arms in brief — each is developed fully in the linked sibling note:

• (A) do_pte_missing (memory.c:3960) — nothing has ever been mapped here. Two lines wide: vma_is_anonymous(vmf->vma) chooses do_anonymous_page (a read installs the shared read-only zero page; a write allocates a fresh zeroed folio) or do_fault (call the VMA’s ->fault op, usually filemap_fault, to bring the page in from the page cache). This is the heart of Demand Paging.
• (B) do_swap_page (memory.c:4200) — the present bit is clear but the entry is not empty, so it encodes something. The non-swap cases are peeled off first (migration, device-exclusive, device-private, hwpoison, PTE marker); what remains is a genuine swap slot. A swap-cache hit is minor; a device read sets VM_FAULT_MAJOR and counts PGMAJFAULT.
• (C) do_numa_page (memory.c:5546) — the page is present and fine; the PTE was deliberately made PROT_NONE by automatic NUMA balancing to provoke exactly this fault and learn which node is touching the page. Minor, and may migrate the folio.
• (D) do_wp_page (memory.c:3655) — a write (or an UNSHARE request) hit a present read-only PTE inside a writable VMA: the copy-on-write break. Exclusively-owned pages are simply made writable in place; otherwise a private copy is allocated. Minor.
• (E) — nothing to fix. The access is already permitted, so the handler sets the accessed bit (and the dirty bit for a write) and returns. A retried fault (FAULT_FLAG_TRIED) skips even the TLB flush.

---

The result: VM_FAULT_* codes and what the arch layer does with them

Every handler returns a bitmask of enum vm_fault_reason (include/linux/mm_types.h:1266-1281). Note the file: mm_types.h holds the return codes and the fault flags; struct vm_fault itself lives in mm.h. It is a bitmask, not an enum value — do_swap_page routinely returns VM_FAULT_MAJOR | VM_FAULT_RETRY — so every consumer tests bits, never equality.

Code	Hex (v6.12)	Kernel-doc description	What it means to the caller
0 (no bit set)	—	—	Resolved. The PTE is installed or fixed; the CPU retries the instruction
VM_FAULT_OOM	0x000001	“Out Of Memory”	User fault: pagefault_out_of_memory() — which at v6.12 handles only memory-cgroup OOM, see below. Kernel fault: exception-table fixup, else oops. Suppressed entirely for a VM_DROPPABLE VMA
VM_FAULT_SIGBUS	0x000002	“Bad access”	do_sigbus() → SIGBUS/BUS_ADRERR. The classic cause is a file truncated out from under a mapping
VM_FAULT_MAJOR	0x000004	“Page read from storage”	Counted as a major fault; also promotes a per-VMA-lock retry to FAULT_FLAG_TRIED
VM_FAULT_HWPOISON	0x000010	“Hit poisoned small page”	do_sigbus() → SIGBUS, machine-check style
VM_FAULT_HWPOISON_LARGE	0x000020	“Hit poisoned large page. Index encoded in upper bits”	Same, plus VM_FAULT_GET_HINDEX() recovers the hstate index from bits 16-19
VM_FAULT_SIGSEGV	0x000040	“segmentation fault”	bad_area_nosemaphore() → SIGSEGV
VM_FAULT_NOPAGE	0x000100	“->fault installed the pte, not return page”	Success. A driver’s ->fault wired the PTE itself and has no struct page to hand back
VM_FAULT_LOCKED	0x000200	“->fault locked the returned page”	Internal contract between ->fault and finish_fault; never reaches the arch layer
VM_FAULT_RETRY	0x000400	“->fault blocked, must retry”	The lock has already been dropped. Re-take it, set FAULT_FLAG_TRIED, replay
VM_FAULT_FALLBACK	0x000800	“huge page fault failed, fall back to small”	Internal; rejoins the small-page path one level down
VM_FAULT_DONE_COW	0x001000	“->fault has fully handled COW”	Internal to do_cow_fault
VM_FAULT_NEEDDSYNC	0x002000	“->fault did not modify page tables and needs fsync() to complete (for synchronous page faults in DAX)”	DAX only
VM_FAULT_COMPLETED	0x004000	“->fault completed, meanwhile mmap lock released”	Return immediately — do not unlock, it is already unlocked
VM_FAULT_HINDEX_MASK	0x0f0000	“mask HINDEX value”	Four bits carrying the hugetlb state index for a poisoned large page

Table: the complete enum vm_fault_reason at v6.12, with the kernel’s own one-line descriptions quoted verbatim. Three of these — LOCKED, DONE_COW, NEEDDSYNC — never escape mm/; the arch layer only ever tests the other twelve. The gaps in the hex values (0x8, 0x80) are retired codes, not reserved space.

VM_FAULT_ERROR is the umbrella macro the arch tail actually tests, defined at mm_types.h:1287-1289 as (VM_FAULT_OOM | VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV | VM_FAULT_HWPOISON | VM_FAULT_HWPOISON_LARGE | VM_FAULT_FALLBACK). The inclusion of VM_FAULT_FALLBACK in an “error” mask is a genuine wart — FALLBACK is not an error anywhere else in the code — but by the time a result reaches do_user_addr_fault the huge-page paths have already consumed and cleared it, so in practice the bit never survives to the arch layer.

flowchart TB
  RES["fault = handle_mm_fault(...)"] --> SIG{"fault_signal_pending(fault, regs)?<br/>RETRY + fatal signal"}
  SIG -->|"yes, user mode"| RETU["return — the signal will be delivered"]
  SIG -->|"yes, kernel mode"| FIX["kernelmode_fixup_or_oops<br/>SIGBUS / BUS_ADRERR"]
  SIG -->|"no"| COMP{"VM_FAULT_COMPLETED?"}
  COMP -->|"yes"| RET2["return — lock already released"]
  COMP -->|"no"| RETRY{"VM_FAULT_RETRY?"}
  RETRY -->|"yes"| AGAIN["flags |= FAULT_FLAG_TRIED<br/>goto retry — re-take mmap_lock"]
  RETRY -->|"no"| UNL["mmap_read_unlock(mm)"]
  UNL --> ERRQ{"fault & VM_FAULT_ERROR?"}
  ERRQ -->|"no"| OK["return — CPU re-executes<br/>the faulting instruction"]
  ERRQ -->|"yes"| FATAL{"fatal signal pending<br/>and kernel mode?"}
  FATAL -->|"yes"| FIX2["kernelmode_fixup_or_oops(0, 0)"]
  FATAL -->|"no"| OOMQ{"VM_FAULT_OOM?"}
  OOMQ -->|"yes, kernel mode"| FIX3["kernelmode_fixup_or_oops<br/>SIGSEGV / SEGV_MAPERR"]
  OOMQ -->|"yes, user mode"| OOMK["pagefault_out_of_memory()<br/>memcg OOM only;<br/>otherwise warn and retry"]
  OOMQ -->|"no"| BUSQ{"SIGBUS or HWPOISON<br/>or HWPOISON_LARGE?"}
  BUSQ -->|"yes"| SB["do_sigbus()"]
  BUSQ -->|"no"| SGQ{"VM_FAULT_SIGSEGV?"}
  SGQ -->|"yes"| SS["bad_area_nosemaphore()"]
  SGQ -->|"no"| BUG["BUG() — unreachable"]

The tail of do_user_addr_fault at v6.12, from the handle_mm_fault return value to the process’s fate. What it shows: four things are tested before the error mask — a pending fatal signal, COMPLETED, RETRY, and the unlock — and only then does the error demultiplex run. The insight to take: the ordering encodes a lock contract. COMPLETED and RETRY both mean “the lock is already gone,” so they must be handled before the mmap_read_unlock(mm) on the normal path; getting that order wrong is a double-unlock, which is exactly the class of bug that made the per-VMA fast path take three kernel releases to stabilise. Note also that a user-mode OOM does not kill the process here — see the paragraph below.

pagefault_out_of_memory() is not the OOM killer. This is the single most commonly mis-stated fact about the fault path. At v6.12 the function is nine lines long (mm/oom_kill.c:1184-1197): it calls mem_cgroup_oom_synchronize(true) and returns if that handled a cgroup OOM; it returns if a fatal signal is already pending; and otherwise it prints a rate-limited pr_warn("Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF") and returns. The comment above it explains the division of labour: the fault handler “ha[s] to take care of the memcg OOM here because this is the only safe context without any locks held but let the oom killer triggered from the allocation context care about the global OOM.” The global OOM killer runs inside the page allocator, not here; a VM_FAULT_OOM reaching the fault handler at all is treated as a leak worth warning about. What happens to the process afterwards is: it returns to userspace, re-executes the faulting instruction, and faults again — a loop that is broken either by memory becoming available or by the allocator-side OOM killer choosing it.

VM_FAULT_RETRY and why faults can be replayed

A fault handler that must sleep — waiting on swap I/O, on a folio lock held by another task, or on a userfaultfd monitor in userspace — faces a dilemma. Holding mmap_lock (or a per-VMA lock) across a multi-millisecond disk read would block every other thread in the process from doing anything structural with its address space. So it does not. It drops the lock, starts the I/O, and returns VM_FAULT_RETRY, and the arch layer replays the whole fault from the top.

The contract is stated in the comment above __handle_mm_fault (mm/memory.c:5810-5814): “On entry, we hold either the VMA lock or the mmap_lock (FAULT_FLAG_VMA_LOCK tells you which). If VM_FAULT_RETRY is set in the result, the mmap_lock is not held on exit.” That single sentence is why the tail of do_user_addr_fault must test RETRY before its mmap_read_unlock.

sequenceDiagram
    autonumber
    participant CPU
    participant AR as do_user_addr_fault
    participant MM as handle_mm_fault
    participant FM as filemap_fault
    participant IO as block layer / swap device

    CPU->>AR: page fault on a cold file page
    AR->>AR: lock_mm_and_find_vma — mmap_lock held (read)
    AR->>MM: handle_mm_fault(flags = DEFAULT)
    MM->>FM: vm_ops->fault
    FM->>FM: filemap_get_folio — miss
    FM->>FM: count PGMAJFAULT, ret = VM_FAULT_MAJOR
    FM->>IO: do_sync_mmap_readahead — start I/O
    Note over FM: maybe_unlock_mmap_for_io()<br/>drops mmap_lock, pins the file
    FM-->>MM: VM_FAULT_MAJOR or VM_FAULT_RETRY
    MM-->>AR: same, and mmap_lock is gone
    AR->>AR: fault_signal_pending()? no
    AR->>AR: flags |= FAULT_FLAG_TRIED
    AR->>AR: goto retry
    AR->>AR: lock_mm_and_find_vma again — VMA may have changed!
    AR->>MM: handle_mm_fault(flags = DEFAULT or TRIED)
    MM->>FM: vm_ops->fault
    Note over FM: FAULT_FLAG_TRIED suppresses<br/>a second async readahead
    IO-->>FM: I/O completed, folio now uptodate
    FM-->>MM: folio + VM_FAULT_LOCKED
    MM->>MM: finish_fault — install the PTE
    MM-->>AR: 0
    AR->>AR: mm_account_fault: TRIED set, so count as MAJOR
    AR->>AR: mmap_read_unlock(mm)
    AR-->>CPU: return — instruction re-executes and succeeds

One major fault, traced across its two passes. What it shows: the first pass exists only to start the I/O and get out of the way; the PTE is installed on the second. The insight to take: three separate mechanisms hang off this. First, FAULT_FLAG_TRIED is what makes the fault count as major even when the second pass finds everything in cache (major = (ret & VM_FAULT_MAJOR) || (flags & FAULT_FLAG_TRIED) at memory.c:5959). Second, mm_account_fault returns early on VM_FAULT_RETRY — “Incomplete faults will be accounted upon completion” — so the fault is counted once, not twice. Third, step 14 is a real hazard: between the two passes another thread may have munmap’d the region, so the retry re-does the VMA lookup from scratch rather than reusing the pointer.

Three more consequences of the retry protocol are worth spelling out:

It is the bridge between the two locking regimes. A fault that returns VM_FAULT_RETRY under FAULT_FLAG_VMA_LOCK does not retry under the VMA lock — it falls through goto lock_mmap to the mmap_lock path. Several handlers exploit this deliberately to opt out of the fast path for cases they cannot do under a per-VMA lock; do_swap_page’s device-private branch is explicit about it: “migrate_to_ram is not yet ready to operate under VMA lock”, so it calls vma_end_read(vma) and returns VM_FAULT_RETRY (memory.c:4231-4238).

A retry under the VMA lock only sets FAULT_FLAG_TRIED if the fault was major. The fast-path code reads if (fault & VM_FAULT_MAJOR) flags |= FAULT_FLAG_TRIED; (fault.c:1347-1348) — a plain fast-path miss does not poison the accounting, so a fault that merely fell back to mmap_lock is still counted as minor. The mmap_lock path, by contrast, sets FAULT_FLAG_TRIED unconditionally on every retry.

Retries can happen more than twice. enum fault_flag’s kernel-doc says so explicitly: “page faults can be allowed to retry for multiple times, in which case we’ll have an initial fault with flags (a) then later on continuous faults with flags (b). We should always try to detect pending signals before a retry to make sure the continuous page faults can still be interrupted if necessary.” That last sentence is why fault_signal_pending() is checked on both retry paths — without it, a process stuck re-faulting on an unresponsive userfaultfd monitor or a wedged device would be unkillable.

---

The retry protocol is a known, unresolved problem

The drop-the-lock-and-replay design solves one problem by creating another, and as of 2026 the kernel community has not settled on a fix. At the 2026 LSFMM+BPF summit Barry Song led a session on exactly this (LWN, Toward better handling of major page faults, Corbet, 22 May 2026). The complaint is the interaction between retry and the fast path: “when satisfying the fault requires initiating I/O, the kernel will release the per-VMA lock then retry the handling of the fault, with mmap_lock held, once the I/O completes. That can create significant mmap_lock contention, causing threads to stall.” In other words, every major fault deliberately gives up the scalability win that per-VMA locking exists to provide.

Four options were weighed, none adopted:

Option	Proposed by	Objection raised
Retry under the VMA lock instead of mmap_lock (patch series posted)	Barry Song	“Would introduce even more complexity into the fault-handling path”
Remove retry entirely and hold the VMA lock across the I/O	Barry Song	Lorenzo Stoakes: more complexity again. Song himself: priority inversion — on Android, “one thread holding mmap_lock could block the high-priority user-interface thread, causing visible stalls,” and it happens “even if the high-priority thread is not accessing the same VMA.” The consensus was that this is a real problem
Apply the VMA-lock retry only for anonymous pages, where it is simple	Matthew Wilcox	Wilcox’s own framing: “the fault-handling code is already too complex now”
Add a separate retry flag meaning “retry under the VMA lock”	Ryan Roberts, Suren Baghdasaryan	Roberts: VM_FAULT_RETRY “is covering too many cases” — it currently conflates “I had to sleep” with “this path cannot run under a VMA lock”

Table: the options discussed for fixing the retry-to-mmap_lock downgrade, at LSFMM+BPF 2026. The session ended with Vlastimil Babka suggesting Song “try the various options to see how they work” — i.e. unresolved. Two data points from the discussion are worth carrying: the contention Song measured was mostly on file-backed pages, not anonymous ones; and a promising cheap fix was raised — when several threads fault the same pages, “the handler could check the up-to-date status of the folio. If it is up to date, then somebody else has already performed the I/O to handle the fault, so the retry can be avoided.”

The practical reading for anyone tuning a system on v6.12: VM_FAULT_RETRY is where per-VMA locking stops helping. A workload whose faults are mostly minor gets the full benefit; a workload that takes many major faults on a shared file mapping falls back to mmap_lock on every one of them, and vma_lock_retry in /proc/vmstat is the counter that shows it.

Kernel faults on user addresses, and the exception table

Not all faults come from user code. The kernel routinely accesses user memory on a process’s behalf — copy_to_user, copy_from_user, get_user, put_user, and the whole user_access_begin() family — and the pointer userspace handed it may be unmapped, unwritable, or outright garbage. Such an access traps exactly like any other page fault, goes through do_user_addr_fault exactly like any other page fault, and is resolved by handle_mm_fault if it can be. The interesting case is when it cannot: the kernel must not die, it must return -EFAULT to the syscall.

This is what the exception table is for. Each instruction that is allowed to fault is registered at build time, via the _ASM_EXTABLE* macros, into a table of struct exception_table_entry sorted by instruction pointer. Each entry carries the faulting instruction’s address, a fixup address to resume at, and a type in its data field.

flowchart TB
  K["kernel executes copy_to_user()<br/>or get_user() on a bad pointer"] --> F["page fault, user address"]
  F --> DUAF["do_user_addr_fault"]
  DUAF --> TRY{"can handle_mm_fault<br/>resolve it?"}
  TRY -->|"yes"| GOOD["PTE installed —<br/>the copy completes normally"]
  TRY -->|"no: bad_area /<br/>VM_FAULT_SIGSEGV / OOM"| UM{"user_mode(regs)?"}
  UM -->|"yes"| SIG["deliver SIGSEGV to the task"]
  UM -->|"no"| KFIX["kernelmode_fixup_or_oops<br/>fault.c:721"]
  KFIX --> FIXUP{"fixup_exception(regs, X86_TRAP_PF, ...)<br/>search_exception_tables(regs->ip)"}
  FIXUP -->|"entry found"| DISPATCH["dispatch on entry type:<br/>EX_TYPE_UACCESS, EX_TYPE_DEFAULT,<br/>EX_TYPE_FAULT, EX_TYPE_BPF,<br/>EX_TYPE_WRMSR / RDMSR, ..."]
  DISPATCH --> RESUME["set regs->ip = fixup address<br/>copy_to_user returns nonzero,<br/>the syscall returns -EFAULT"]
  FIXUP -->|"no entry"| PF{"is_prefetch()?<br/>AMD erratum 91"}
  PF -->|"yes"| IGN["ignore, resume"]
  PF -->|"no"| OOPS["page_fault_oops — fault.c:641"]
  OOPS --> SPECIAL{"special cases first"}
  SPECIAL --> SO["vmalloc guard page hit:<br/>handle_stack_overflow on the<br/>double-fault stack"]
  SPECIAL --> EFI["CONFIG_EFI:<br/>efi_crash_gracefully_on_page_fault"]
  SPECIAL --> KF["not-present fault:<br/>kfence_handle_page_fault"]
  SPECIAL --> DIE["oops_begin / __die / oops_end<br/>prints 'CR2: %016lx'"]

How a bad pointer inside the kernel becomes either -EFAULT or an oops, at v6.12. What it shows: the exception table is consulted only after the ordinary fault machinery has given up, and the entry’s presence is what distinguishes “an instruction we knew might fault” from “a kernel bug.” The insight to take: this is the whole mechanism behind copy_to_user()’s error return. There is no pointer validation in copy_to_user — it simply dereferences, and the exception table converts the resulting trap into a return value. That is why copy_to_user is fast and why access_ok() checks only the range, never the mapping.

kernelmode_fixup_or_oops (fault.c:721) is three steps: WARN_ON_ONCE(user_mode(regs)) (it must never be reached from user mode), then fixup_exception(), then — before giving up — is_prefetch(), which swallows the spurious fault AMD erratum #91 produces on a PREFETCH instruction. Only then page_fault_oops.

fixup_exception (arch/x86/mm/extable.c:300) looks the instruction pointer up with search_exception_tables(regs->ip), and if it finds an entry, switches on the type stored in e->data. The types are not cosmetic — EX_TYPE_UACCESS additionally warns if the trap was a #GP on a non-canonical address (“General protection fault in user access. Non-canonical address?”), EX_TYPE_FPU_RESTORE handles a corrupted XRSTOR state, EX_TYPE_BPF unwinds a JIT-compiled BPF program, and EX_TYPE_WRMSR/EX_TYPE_RDMSR are what make wrmsrl_safe() safe. All of them ultimately do the same core thing: set regs->ip to the fixup address so execution resumes at the error path the compiler laid out.

page_fault_oops (fault.c:641) is worth reading for its three pre-death special cases, each of which exists because dying naively would make debugging impossible:

• Kernel stack overflow. If CONFIG_VMAP_STACK is on and the faulting address is in vmalloc space and get_stack_guard_info() recognises it as a stack guard page, the handler switches to the double-fault IST stack (call_on_stack(__this_cpu_ist_top_va(DF) …)) before reporting. The comment explains why: “We’re likely to be running with very little stack space left… we don’t want to make it all the way into the oops code and then double-fault, because we’re likely to break the console driver and lose most of the stack dump.”
• EFI runtime services. “Buggy firmware could access regions which might page fault” — efi_crash_gracefully_on_page_fault(address) gets a chance to avoid hanging the machine.
• KFENCE. For a not-present fault only (!(error_code & X86_PF_PROT)), kfence_handle_page_fault() may claim it: KFENCE deliberately leaves guard pages unmapped around sampled allocations, so a fault there is a caught bug report, not a crash.

If none of those apply it calls oops_begin(), show_fault_oops(), __die("Oops", …), and prints one last line — printk(KERN_DEFAULT "CR2: %016lx\n", address) — described in the source as the “executive summary in case the body of the oops scrolled away.” That CR2: line at the bottom of a kernel oops is the faulting address, and it comes from here.

---

Other ways into handle_mm_fault — and when they are the right tool

The hardware trap is the usual entry into handle_mm_fault, but it is not the only one, and the alternatives matter because they behave differently in ways that surprise people reading /proc counters or debugging latency.

flowchart LR
  subgraph HW["hardware-initiated"]
    T["CPU trap, vector 14"] --> DUAF["do_user_addr_fault<br/>regs != NULL"]
  end
  subgraph SW["software-initiated (synthetic faults)"]
    G["get_user_pages / pin_user_pages"] --> FI["faultin_page<br/>gup.c:1148"]
    FUF["fixup_user_fault<br/>gup.c:1623<br/>(futex, atomic-context retry)"] --> HMF
    POP["MAP_POPULATE, mlock,<br/>MADV_POPULATE_READ/WRITE"] --> PVR["populate_vma_page_range<br/>gup.c:1876"]
    PVR --> FI
    FI --> HMF
  end
  DUAF --> HMF["handle_mm_fault<br/>memory.c:6042"]
  HMF --> HT{"is_vm_hugetlb_page?"}
  HT -->|"yes"| HF["hugetlb_fault<br/>hugetlb.c:6294<br/>separate handler entirely"]
  HT -->|"no"| MHM["__handle_mm_fault"]
  MHM --> UFF{"userfaultfd registered<br/>on this VMA?"}
  UFF -->|"yes"| HU["handle_userfault<br/>fault handed to a<br/>userspace monitor"]
  UFF -->|"no"| PTE["handle_pte_fault"]

The entry points into handle_mm_fault at v6.12, and the two places control can leave the ordinary path. What it shows: synthetic faults from get_user_pages reach exactly the same code, but with regs == NULL. The insight to take: regs == NULL is load-bearing. mm_account_fault uses it to decide whether to emit perf events — “If the fault is done for GUP, regs will be NULL. We only do the accounting for the per thread fault counters who triggered the fault, and we skip the perf event updates.” So a page faulted in by get_user_pages does bump min_flt/maj_flt and PGFAULT, but does not appear in perf stat -e page-faults. If your perf numbers and your /proc/<pid>/stat numbers disagree, this is usually why.

Entry point	Who uses it	How it differs from a hardware fault	When to prefer it
do_user_addr_fault	The CPU	The baseline. Full flags, perf events, per-VMA fast path	You do not choose this; it chooses you
faultin_page (gup.c:1148)	get_user_pages, pin_user_pages, direct I/O, vmsplice, KVM	regs == NULL (no perf events); may set FAULT_FLAG_REMOTE for another task’s mm; can set FAULT_FLAG_RETRY_NOWAIT, which is illegal under the per-VMA lock	The kernel needs the struct page, not just a valid mapping
fixup_user_fault (gup.c:1623)	Chiefly the futex code	“This function will not return with an unlocked mmap_lock”, unlike filemap_fault. Unconditionally calls handle_mm_fault so that software-maintained dirty/young bits are updated — which plain GUP does not guarantee	You touched user memory inside pagefault_disable(), got -EFAULT, and want to resolve it and retry
populate_vma_page_range (gup.c:1876)	MAP_POPULATE, mlock/mlockall, MADV_POPULATE_READ/WRITE	Loops over a whole range, one synthetic fault per page, up front	You want the faults to happen now, at a moment you control, instead of during a latency-sensitive phase. See Demand Paging for the trade-off
hugetlb_fault (hugetlb.c:6294)	Any MAP_HUGETLB / hugetlbfs VMA	A completely separate handler with its own reservation accounting and locking. None of handle_pte_fault’s dispatch applies	Not a choice — it is implied by the mapping type
handle_userfault	userfaultfd(2) monitors	The fault is sent to userspace and the thread blocks until a monitor resolves it with UFFDIO_COPY/UFFDIO_CONTINUE	Live migration, distributed shared memory, on-demand paging from a userspace store, snapshot restore

Table: the six ways a page fault gets resolved at v6.12. The row that changes system design most is the last one — userfaultfd turns the kernel’s “fill this page” decision into an upcall, which is how post-copy live migration works: the destination VM starts running immediately and every fault on not-yet-transferred memory is answered by pulling that page over the network.

Two smaller variants round out the picture. KVM asynchronous page faults reuse vector 14 to tell a guest that the host has the page swapped out, letting the guest schedule another task rather than spin — handled and consumed in exc_page_fault before handle_page_fault ever runs. And MADV_POPULATE_READ/WRITE (Linux 5.14 and later, per madvise(2)) differ from MAP_POPULATE in a way worth knowing: they “do not hide errors, can be applied to (parts of) existing mappings,” and on failure return an error rather than raising SIGBUS. MAP_POPULATE silently ignores failures.

Observing the fault path

Everything in this note is directly observable on a running system, and the instrumentation is unusually good.

Static tracepoints on the arch entry. trace_page_fault_entries (fault.c:1456) fires one of two events depending on user_mode(regs), both defined in arch/x86/include/asm/trace/exceptions.h and both printing address, ip and the raw error_code:

# every user page fault, with the raw hardware error code
perf trace -e exceptions:page_fault_user
# or via ftrace
echo 1 > /sys/kernel/tracing/events/exceptions/page_fault_user/enable
cat /sys/kernel/tracing/trace_pipe
# TP_printk is: "address=%ps ip=%ps error_code=0x%lx"

Decode the error_code with the bit table above: 0x6 is a user-mode write to a not-present page (WRITE|USER), 0x7 is a user-mode write to a present page — a COW break — and 0x14 is a user instruction fetch of a not-present page (INSTR|USER), i.e. demand-paging an executable.

Perf software events. PERF_COUNT_SW_PAGE_FAULTS is emitted by the arch handler at entry, before the fault is known to succeed; PERF_COUNT_SW_PAGE_FAULTS_MIN and _MAJ are emitted by mm_account_fault at the end, and only for faults that neither retried nor errored.

perf stat -e page-faults,minor-faults,major-faults ./workload
perf record -e minor-faults -g ./workload   # which call sites fault the most

Per-VMA lock effectiveness. If the kernel was built with CONFIG_PER_VMA_LOCK_STATS (bool "Statistics for per-vma locks", depends on PER_VMA_LOCK, default N, described as introducing “a small overhead in the page fault path”), four counters appear in /proc/vmstat — vma_lock_success, vma_lock_abort, vma_lock_retry, vma_lock_miss (mm/vmstat.c:1437-1440). A high vma_lock_abort relative to vma_lock_success means threads are faulting while another thread mutates the same VMA, and the fast path is buying nothing.

mmap_lock contention. mm/mmap_lock.c defines three tracepoints — mmap_lock_start_locking, mmap_lock_acquire_returned, mmap_lock_released — which, with CONFIG_MEMCG, attribute each acquisition to a cgroup path. These are the right tool for confirming that a latency spike really is address-space lock contention rather than, say, reclaim.

Global counters. /proc/vmstat’s pgfault and pgmajfault (mm/vmstat.c:1298-1299) are the system-wide totals; pgreuse counts write faults resolved by reusing an exclusively-owned page instead of copying. Per-process, /proc/<pid>/stat fields minflt and majflt, and getrusage(2)’s ru_minflt/ru_majflt — all of which Minor and Major Faults covers properly.

Failure Modes and Common Misunderstandings

• “A page fault means a bug.” No — on Linux the overwhelming majority of faults are expected: first touch of mmap’d memory, COW break after fork, swap-in, page-cache fill. A fault is the normal way the kernel populates page tables (see Demand Paging). Only access_error / VM_FAULT_SIGSEGV faults are bugs.
• “SIGSEGV vs SIGBUS.” SIGSEGV means the address is not in any (accessible) VMA, or the access violates the VMA’s permissions — access_error or VM_FAULT_SIGSEGV. SIGBUS means the address is in a valid VMA but the backing object cannot supply the page. The canonical cause is visible in filemap_fault’s very first lines: max_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); if (index >= max_idx) return VM_FAULT_SIGBUS; (mm/filemap.c:3313-3315) — the file was truncated so the offset no longer exists. The check is repeated after taking the folio lock, because i_size can shrink under you mid-fault. Hardware-poisoned pages (VM_FAULT_HWPOISON) also become SIGBUS.
• “pagefault_out_of_memory() invokes the OOM killer.” It does not, and has not for some years. At v6.12 it handles memory-cgroup OOM only and otherwise prints "Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF". The global OOM killer lives in the allocator. See the VM_FAULT_* section above.
• “The per-VMA lock makes faults lockless.” It does not — the VMA is genuinely locked, just per-VMA instead of per-mm. The win is that disjoint faults stop contending; a fault still serialises against a writer of the same VMA, and on any complication it falls back to mmap_lock. Nor is it universally compiled in: it requires ARCH_SUPPORTS_PER_VMA_LOCK && MMU && SMP, and on x86 that symbol is selected by config X86_64, not config X86.
• “Per-VMA locks are available in all builds now.” Not as of v7.3-rc2. That change is an April-2026 patch posting, not merged code — see the version table above, which was established by reading mm/Kconfig at six successive tags.
• “Stack growth is special-cased.” Yes, and more expensively than most people expect. A fault just below a VM_GROWSDOWN VMA’s vm_start makes lock_mm_and_find_vma drop its read lock, take the write lock, re-look-up the VMA, call expand_stack_locked(), and downgrade back to a read lock — because mmap_upgrade_trylock() is a stub returning false at v6.12. Deep recursion therefore serialises every thread in the process against the address-space write lock, once per stack page. The per-VMA fast path cannot help here at all.
• “A fault under the per-VMA lock behaves identically.” It does not. Several handlers explicitly refuse to run under FAULT_FLAG_VMA_LOCK and return VM_FAULT_RETRY so the fault is replayed under mmap_lock — do_swap_page’s device-private branch is the clearest example. A workload that hits those paths gets zero benefit from per-VMA locking, and vma_lock_retry in /proc/vmstat is how you find out.
• “copy_to_user validates the pointer.” It validates the range (access_ok), never the mapping. A bad pointer faults, and the exception table turns the trap into a nonzero return. This is why a copy_to_user to an unmapped address is cheap-ish and safe, while the same dereference in kernel code with no _ASM_EXTABLE entry is an oops.
• “Perf’s page-faults and /proc/<pid>/stat’s minflt should agree.” They need not. Faults driven by get_user_pages (direct I/O, vmsplice, KVM, RDMA registration) pass regs == NULL, so mm_account_fault updates the task counters but deliberately skips the perf events. Also, PERF_COUNT_SW_PAGE_FAULTS is emitted on entry and counts faults that later fail, while PGFAULT counts both successes and failures but _MIN/_MAJ count only successes.
• “A retried fault is counted twice.” No. mm_account_fault returns immediately on VM_FAULT_RETRY — “Incomplete faults will be accounted upon completion” — and the completing pass carries FAULT_FLAG_TRIED, which forces the single accounting to be major. That is why a fault can be counted major even though its final pass found everything in cache.
• Spurious faults. On SMP, a stale TLB entry can make a CPU fault on a PTE that is perfectly fine. handle_pte_fault’s case (E) calls flush_tlb_fix_spurious_fault and returns success; the kernel-address side has spurious_kernel_fault, which re-walks the real page table and silently returns. Neither does any real work, but both still cost a trap — a workload with heavy concurrent mprotect traffic can burn measurable time here.
• is_prefetch and the AMD errata. Two separate AMD errata produce faults that are not faults: #93 corrupts RIP so a kernel instruction fetch looks like a user address (checked in the very first early reject), and #91 produces a spurious fault on a PREFETCH instruction (checked in kernelmode_fixup_or_oops right before oopsing). If you are reading a confusing oops on old AMD hardware, these are worth remembering.

Production Notes

Fault rate is a first-class latency signal, and the two classes mean different things. A high minor fault rate is usually a workload shape — a process building a large heap, or a fork-heavy service paying COW. A high major fault rate means the working set does not fit and the system is reading from disk to make progress; that is the classic thrashing signature, and Pressure Stall Information quantifies the resulting stall time better than the raw counters do. The counters themselves are the subject of Minor and Major Faults.

The mmap_lock bottleneck was real, and the fix is measurable. Per Liam Howlett’s 2024 LSFMM+BPF report, fault handling got slower between 5.19 and 6.2 as the groundwork went in — “distributors were starting to get nervous” — then doubled in 6.4 when per-VMA locking landed, and was “almost three times better than it had been before the work began” by 6.6 (LWN, Faster page faults with RCU-protected VMA walks). If you are running a heavily multi-threaded, fault-heavy workload on a pre-6.4 kernel, that is a concrete reason to upgrade.

The feature needed three releases to become trustworthy. Per-VMA locking merged in 6.4 and immediately produced a fork()-versus-fault data-corruption bug, plus interactions with the StackRot stack-expansion fix; both were resolved by forcing explicit per-VMA write-locking, landing in 6.4.3 in July 2023 (LWN, Stabilizing per-VMA locking). The general lesson is worth carrying: fine-grained locking in the fault path is exactly the kind of change whose bugs are rare, data-corrupting, and invisible in testing.

Pre-faulting is the standard mitigation, and it is not free. Latency-sensitive services — databases, JVMs with large heaps, low-latency trading systems, real-time audio — routinely remove fault-driven jitter from their hot path by pre-populating: MAP_POPULATE at mmap time, MADV_POPULATE_READ/WRITE on an existing mapping, or mlockall(MCL_CURRENT|MCL_FUTURE) to pre-fault and pin. The cost is a higher resident footprint and a longer, less predictable startup. Demand Paging develops this trade-off properly.

Reading a fault-related oops. The CR2: line at the bottom of the oops is the faulting address, printed deliberately last as an “executive summary in case the body of the oops scrolled away.” The error code appears in show_fault_oops’s decoded form (“reserved bit violation”, etc.). A CR2 of a small value such as 0x0000000000000008 is a NULL-pointer-plus-offset dereference; a CR2 in vmalloc space on a CONFIG_VMAP_STACK kernel is very likely a kernel stack overflow, which is why page_fault_oops switches to the double-fault stack before reporting it.

When the fast path is not helping. If CONFIG_PER_VMA_LOCK_STATS is available, compare vma_lock_success against vma_lock_abort + vma_lock_retry in /proc/vmstat. Persistent aborts mean concurrent structural changes to the same VMAs — a workload doing continuous mmap/munmap/mprotect while faulting, which no amount of lock granularity will fix. The remedy is at the application layer: pool and reuse mappings rather than churning them.

Architecture portability. Everything in this note above handle_mm_fault is x86-64-specific: the vector number, CR2, the X86_PF_* bits, SMAP, the vsyscall carve-out, the errata. Everything from handle_mm_fault down is shared by every architecture. When porting knowledge between architectures, FAULT_FLAG_* and VM_FAULT_* are the stable vocabulary; error_code is not.

See Also

• Demand Paging — the policy of lazy population that this handler implements. The boundary: that note owns what the kernel chooses to populate and when (demand-zero vs demand-fill vs swap-in, fault-around, the pre-faulting knobs); this note owns the code path a single fault takes (the trap, the locks, the permission gate, the dispatch, the return protocol). If your question is “why is memory not allocated until I touch it,” go there; if it is “what happens between the trap and the PTE being written,” stay here.
• Minor and Major Faults — the classification and accounting (maj_flt/min_flt, PGFAULT/PGMAJFAULT, /proc, getrusage); this note routes faults there for counting and does not re-derive the rules.
• Copy-on-Write and fork — dispatch arm (D), do_wp_page, in full depth, including Dirty COW and the GUP-after-fork saga.
• Anonymous vs File-Backed Memory — the family split that do_pte_missing makes, and the reverse-mapping and LRU consequences.
• The Zero Page and Lazy Allocation — the read-fault half of do_anonymous_page.
• Virtual Memory Areas · The Maple Tree and VMA Lookup · The mm_struct and Process Address Space — what the handler looks up and what lock_vma_under_rcu walks.
• Split Page Table Locks · Linux Page Table Hierarchy · Page Table Entries and Hardware Flags · Page Table Entry — the structures __handle_mm_fault allocates, walks and locks.
• The Page Cache · The Page Cache and address_space · Readahead — where do_fault fills file pages from, and why a cache miss is declared a major fault before any I/O completes.
• Linux Swap Subsystem · The Swap Cache — dispatch arm (B).
• Automatic NUMA Balancing — dispatch arm (C), and why a perfectly good PTE is deliberately made PROT_NONE.
• Transparent Huge Pages — the create_huge_pmd / create_huge_pud branches inside __handle_mm_fault and the VM_FAULT_FALLBACK protocol.
• The Translation Lookaside Buffer and TLB Shootdowns — why spurious faults exist and what flush_tlb_fix_spurious_fault is for.
• Memory Overcommit and Accounting · The OOM Killer — where VM_FAULT_OOM actually gets resolved (not here).
• Direct Reclaim — what happens inside the allocator when do_anonymous_page cannot get a folio.
• Multi-Generational LRU — what lru_gen_enter_fault is telling the page-aging machinery.
• Memory Management Unit — the hardware that raises the trap in the first place.
• UP: Linux Memory Management MOC §3 — The Page-Fault Handler and Demand Paging.