vault

Transparent Huge Pages

14 min read

Transparent Huge Pages

Transparent Huge Pages (THP) is the Linux kernel mechanism that automatically and opportunistically backs a process’s virtual memory with huge pages — large physically-contiguous, naturally-aligned page-table mappings (classically the PMD-sized 2 MiB page on x86-64, alongside 4 KiB base pages) — without the application having to reserve them up front the way hugetlbfs demands. The kernel either allocates a huge page directly in the page-fault path when a mapping is large and aligned, or lets the [[khugepaged and THP Collapse|khugepaged]] daemon promote runs of base pages into huge pages after the fact (per the v6.18 admin guide). The payoff is fewer Translation Lookaside Buffer (TLB) misses — one TLB entry covers 2 MiB instead of 4 KiB — and fewer page faults; the cost is allocation latency, internal fragmentation (a 2 MiB page that only touches one byte wires up 2 MiB), and the splitting/collapse machinery needed to keep the illusion graceful. As of the 6.12 LTS (2024-11-17) and 6.18 LTS (2025-11-30) kernels, THP works for anonymous memory and tmpfs/shmem only, and now also supports multi-size THP (mTHP) — PTE-mapped huge pages smaller than the PMD size.

THP is one of two ways to use huge pages in Linux; see Huge Pages Overview for how it contrasts with the explicit-reservation hugetlbfs route, and Compound Pages and Large Folios for the struct folio machinery that physically represents a huge page.

Mental Model

Think of THP as the kernel quietly upgrading your memory behind your back whenever it can, and gracefully downgrading it whenever something can’t cope with a huge page. A 2 MiB region that is mapped, aligned, and faulted in one go gets a single huge Page Middle Directory (PMD) entry — one TLB slot maps the whole 2 MiB. The moment any code path that doesn’t understand huge PMDs touches part of that region (an mprotect() on half of it, a partial munmap(), a swap-out), the kernel splits the huge mapping back down to 512 ordinary Page Table Entries (PTEs), each pointing at a 4 KiB sub-page, and everything keeps working. This “graceful fallback” is the core design principle of THP (design doc): a huge page is always something the kernel can fall back from.

flowchart TD
  FAULT["Page fault on a large,<br/>aligned anon/shmem region"] -->|"defrag allows it,<br/>huge page available"| HUGE["PMD-mapped 2 MiB THP<br/>(1 TLB entry covers 2 MiB)"]
  FAULT -->|"huge page not<br/>immediately available"| BASE["512 x 4 KiB base pages<br/>(PTE-mapped)"]
  BASE -->|"khugepaged scans &<br/>collapses later"| HUGE
  HUGE -->|"mprotect / munmap part,<br/>swap-out, GUP pin"| SPLIT["split_huge_pmd /<br/>split_huge_page"]
  SPLIT --> BASE
  BASE -->|"mTHP: medium order<br/>(16K..1M), PTE-mapped"| MTHP["multi-size THP<br/>(contiguous PTEs)"]

The THP life-cycle as a two-way street. What it shows: a fault on a suitable region either gets a PMD-sized huge page immediately, or falls back to base pages that khugepaged may later collapse upward; any operation that can’t handle a huge PMD splits it back down. mTHP is a middle tier — contiguous runs of PTEs smaller than 2 MiB. The insight: THP is never a one-way commitment. Every promotion has a corresponding demotion path, which is exactly why THP can be “transparent” — no caller is ever stuck holding a huge page it can’t deal with.

Why Huge Pages Help — the Mechanism

The kernel doc spells out two effects (v6.18 admin guide). The first, minor, effect is fault reduction: touching a 2 MiB region takes a single fault instead of 512, cutting enter/exit-kernel frequency by a factor of 512 — but only the first time the memory is accessed, so it is “almost completely irrelevant” for long-running workloads, and it carries a downside (a larger clear-page/copy-page on each fault). The second, dominant, effect lasts the whole runtime: (1) a TLB miss resolves faster because the hardware page-table walk is shorter, and (2) a single TLB entry maps 2 MiB instead of 4 KiB, so the finite TLB covers 512× more memory and misses far less often. This matters even more under virtualization with nested page tables, where a TLB miss is otherwise especially expensive — and the speedup compounds if both the host (KVM) and guest use huge pages.

The fundamental scarce resource being defended here is TLB reach — the total memory the TLB can map at once. This is the same pressure that motivates hugetlbfs and large folios; see The Translation Lookaside Buffer and TLB Shootdowns for why the TLB is small and a miss is costly.

Where a huge page comes from: the GFP semantics

A THP allocation is just a high-order request to the buddy allocator — order-9 (2^9 = 512 pages) for a 2 MiB PMD page. The crucial subtlety is how hard the kernel tries. The __GFP_* flags (GFP Flags and Allocation Contexts) differ between the fault path and khugepaged (gfp_types.h, v6.12):

#define GFP_TRANSHUGE_LIGHT  ((GFP_HIGHUSER_MOVABLE | __GFP_COMP | \
             __GFP_NOMEMALLOC | __GFP_NOWARN) & ~__GFP_RECLAIM)
#define GFP_TRANSHUGE        (GFP_TRANSHUGE_LIGHT | __GFP_DIRECT_RECLAIM)

GFP_TRANSHUGE_LIGHT masks out __GFP_RECLAIM: it will not trigger direct reclaim or compaction and fails fast if a contiguous 2 MiB run is not already free. The kernel comment states it is “by default used in page fault path.” GFP_TRANSHUGE re-adds __GFP_DIRECT_RECLAIM, so it can stall to reclaim and compact memory; it is “used by khugepaged.” This is the mechanical reason the fault path is cheap-or-nothing while collapse can afford to work harder — and the reason a fragmented system silently stops getting fault-time THP even though khugepaged keeps producing them.

The enabled Policy: always / madvise / never

The headline global control is /sys/kernel/mm/transparent_hugepage/enabled, which takes one of three values (v6.18 admin guide):

echo always  >/sys/kernel/mm/transparent_hugepage/enabled
echo madvise >/sys/kernel/mm/transparent_hugepage/enabled
echo never   >/sys/kernel/mm/transparent_hugepage/enabled
  • always — every eligible anonymous mapping is a THP candidate, system-wide. Maximum TLB benefit, maximum risk of memory bloat (the doc’s own warning: an app that mmaps a large region but touches one byte may get a full 2 MiB page “for no good”).
  • madvise — THP is used only in regions a process explicitly opts in via madvise(addr, len, MADV_HUGEPAGE). This is the conservative default on many distributions.
  • never — THP disabled for anonymous memory (mostly for debugging or for latency-sensitive workloads).

A critical 6.18 clarification (added relative to 6.12): setting never everywhere does not globally disable THP, because madvise(..., MADV_COLLAPSE) ignores these knobs and collapses unconditionally. To truly bar THP for one process, use the per-process prctl() below.

The boot-time default can be set on the kernel command line with transparent_hugepage=always|madvise|never.

The companion: defrag

/sys/kernel/mm/transparent_hugepage/defrag controls how aggressively the kernel reclaims/compacts to manufacture a huge page when one is not immediately free. Its five modes (v6.18 admin guide):

  • always — a faulting app stalls in direct reclaim + compaction to get a THP now (good for VMs willing to delay startup).
  • defer — wake [[kswapd and Background Reclaim|kswapd]] and kcompactd in the background; let khugepaged install the THP later.
  • defer+madvise — direct reclaim/compaction like always but only for MADV_HUGEPAGE regions; everyone else gets the defer treatment.
  • madvise — direct reclaim only for MADV_HUGEPAGE regions. This is the documented default.
  • never — never defrag; fall back to base pages unless a huge page is sitting free.

Note the layering: enabled decides whether a region is a candidate; defrag decides how hard to work to satisfy a candidate. With the common enabled=madvise/defrag=madvise pair, a non-advised mapping never gets a fault-time THP at all, and an advised one will stall to reclaim for it.

Per-Process Control: prctl(PR_SET_THP_DISABLE)

New in 6.18 relative to 6.12 is documented per-process THP control via prctl(2), inherited across fork(2) and execve(2) (v6.18 admin guide):

prctl(PR_SET_THP_DISABLE, 1, 0, 0, 0);                          /* disable THP entirely, even MADV_COLLAPSE */
prctl(PR_SET_THP_DISABLE, 1, PR_THP_DISABLE_EXCEPT_ADVISED, 0, 0); /* disable except where advised */
prctl(PR_SET_THP_DISABLE, 0, 0, 0, 0);                          /* re-enable */

The first form is the only way to truly turn THP off for a process, because (as noted) MADV_COLLAPSE overrides the sysfs never. PR_GET_THP_DISABLE returns a bitmask: 0 = no override, 1 = fully disabled, 3 = disabled-except-advised.

Uncertain

Verify: the exact kernel release that introduced PR_SET_THP_DISABLE’s PR_THP_DISABLE_EXCEPT_ADVISED mode and prctl-based THP control as documented. Reason: it is present in the 6.18 transhuge.rst but absent from the 6.12 copy I fetched, so it landed between 6.12 and 6.18; I did not pin the precise merge release. To resolve: git log the Documentation/admin-guide/mm/transhuge.rst “process THP controls” section against the v6.12..v6.18 range. uncertain

Multi-Size THP (mTHP)

The big modern shift, present in both 6.12 and 6.18, is multi-size THP (mTHP): the ability to back anonymous memory (and tmpfs/shmem) with huge pages that are bigger than a base page but smaller than the PMD size — 16 KiB, 32 KiB, 64 KiB, 128 KiB, and so on in power-of-two page counts (v6.18 admin guide). Anonymous mTHP and its per-size sysfs interface first appeared in Linux 6.8 (early 2024): the transhuge.rst shipped with the v6.8 tag documents “multi-size THP (mTHP)” and the hugepages-<size>kB/enabled knobs, whereas the v6.7 tag contains neither — so 6.8 is the introducing release, and the feature is fully present and stable across both 6.12 and 6.18 LTS. Unlike the classic PMD-sized THP, an mTHP stays PTE-mapped — it is a contiguous run of ordinary PTEs, represented internally as a large folio. The benefit profile is a deliberate compromise: page faults drop by the order factor (4×, 8×, 16×…) and some architectures (notably ARM with its “contiguous PTE” hint) can TLB-compress contiguous, aligned PTEs into fewer entries, but the latency spikes and clear/copy costs are far smaller than a full 2 MiB page, and there is less wasted memory on a sparsely-touched region.

Each supported size has its own sysfs directory; the policy knob is identical in spirit to the global one but adds an inherit mode:

echo always  >/sys/kernel/mm/transparent_hugepage/hugepages-64kB/enabled
echo madvise >/sys/kernel/mm/transparent_hugepage/hugepages-64kB/enabled
echo never   >/sys/kernel/mm/transparent_hugepage/hugepages-2048kB/enabled
echo inherit >/sys/kernel/mm/transparent_hugepage/hugepages-2048kB/enabled

The default arrangement, stated verbatim by the doc: “PMD-sized hugepages have enabled=“inherit” and all other hugepage sizes have enabled=“never”.” So out of the box only the PMD size is active (following the top-level enabled), and you must explicitly opt smaller sizes in. If multiple sizes are enabled, “the kernel will select the most appropriate enabled size for a given allocation.” Boot-time configuration uses thp_anon=<size>[KMG]...:<state>, e.g. thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never; if thp_anon= is given at all, any unlisted size defaults to never. The thp_anon= boot parameter itself is newer than the sysfs interface — it first appears in the v6.12 transhuge.rst and is absent from v6.8–v6.11.

khugepaged, importantly, only collapses to PMD-sized THP — it makes no attempt to produce mTHP sizes (note in the admin guide). mTHP therefore comes only from the fault path, not from background collapse. See khugepaged and THP Collapse.

tmpfs and shmem THP

THP also backs tmpfs/shmem (shared memory: SysV SHM, memfd, MAP_ANONYMOUS|MAP_SHARED, GPU DRM objects). A tmpfs mount takes a huge= mount option with values always, never, within_size (only if the page fits fully within i_size), and advise (only on madvise() hint). The internal shmem mount is controlled by /sys/kernel/mm/transparent_hugepage/shmem_enabled and per-size shmem_enabled knobs, which additionally accept force/deny (testing artifacts) on the global knob. As of 6.18 the tmpfs default policy is settable at boot via transparent_hugepage_tmpfs=<policy> and defaults to never. Anonymous-file (regular filesystem) THP beyond shmem is not yet supported — “in the future it can expand to other filesystems.”

Splitting — the Graceful-Fallback Machinery

A huge page must be splittable for THP to be transparent. There are two split operations, both implemented in mm/huge_memory.c (v6.12) and described by the design doc:

  • split_huge_pmd(vma, pmd, addr) (__split_huge_pmd in the source) — splits only the page-table mapping: the one huge PMD entry becomes a table of 512 PTEs all pointing into the same physical huge folio. The physical page is untouched; this never fails. It is what fires on mprotect() or partial munmap() of a huge region (counter: thp_split_pmd in /proc/vmstat).
  • split_huge_page(page) (split_huge_page_to_list_to_order in the source) — splits the physical folio into individual base pages, redistributing the head folio’s refcounts to the now-independent pages. This is what the VM does before swapping a THP out. It can fail if the page is pinned (e.g. by [[get_user_pages and Page Pinning|get_user_pages]]): can_split_folio() expects the page count to equal the sum of sub-page mapcounts plus the caller’s expected pins, and a stray GUP pin breaks that invariant. Counters: thp_split_page, thp_split_page_failed.

Partial unmap does not split immediately. Instead the kernel queues the folio on a deferred split list via deferred_split_folio() (also in huge_memory.c), and the actual split happens later under memory pressure through the shrinker interface (counter thp_deferred_split_page). 6.18 adds an underused-THP shrinker: a THP whose number of zero-filled sub-pages exceeds max_ptes_none is “underused” and gets split under pressure; toggle with /sys/kernel/mm/transparent_hugepage/shrink_underused (counter thp_underused_split_page). This directly attacks the memory-bloat failure mode.

Monitoring

Per-system PMD-THP usage is in /proc/meminfo: AnonHugePages (anonymous PMD-mapped THP — the doc notes it “should have been called AnonHugePmdMapped”), ShmemHugePages/ShmemPmdMapped, and FilePmdMapped. Per-process, sum the matching fields in /proc/<pid>/smaps (expensive — read sparingly). /proc/vmstat carries the success/fallback counters: thp_fault_alloc / thp_fault_fallback (fault path), thp_collapse_alloc / thp_collapse_alloc_failed (khugepaged), thp_swpout / thp_swpout_fallback, plus the split counters above. Per-size mTHP statistics live under /sys/kernel/mm/transparent_hugepage/hugepages-<size>kB/stats/ with files like anon_fault_alloc, anon_fault_fallback, swpout, split, nr_anon, and nr_anon_partially_mapped.

Failure Modes and the Famous “Disable THP” Advice

The defining hazard of enabled=always is memory bloat from internal fragmentation: a sparsely-used large mapping gets fully-populated 2 MiB pages, and MADV_DONTNEED/madvise(MADV_FREE) on sub-page granularity forces PMD splits that churn. The kernel doc itself flags this (the “2 MB mapping that touches 1 byte” example), and the madvise(2) man page repeats it (madvise.2): THP “can very easily waste memory.”

The second hazard is latency: with defrag=always, a fault can stall the faulting thread in synchronous compaction for a contiguous 2 MiB run — an unpredictable multi-millisecond pause. This is why memory-allocator-heavy, latency-sensitive servers historically recommended enabled=never or at least defrag=never/madvise. Databases that manage their own memory and dislike both the bloat and the stalls (MongoDB, Redis, and others) long advised disabling THP outright; the mechanism behind that advice is exactly the GFP __GFP_DIRECT_RECLAIM stall and the bloat described above.

Uncertain

Verify: current vendor recommendations (e.g. MongoDB, Redis) on disabling THP as of 2026, and whether mTHP + the underused-THP shrinker have softened that advice. Reason: the vendor pages I attempted to fetch were JavaScript-rendered and did not yield clean text, and recommendations evolve. To resolve: re-fetch the live MongoDB/Redis production-notes pages and the relevant LWN coverage. uncertain

mTHP plus shrink_underused are the kernel’s modern answer to the bloat-and-latency complaint: medium-order pages have far smaller clear/copy costs and waste less on sparse regions, and the underused shrinker reclaims zero-filled THP sub-pages under pressure — reducing the blast radius that made blanket enabled=never the safe default.

Alternatives — THP vs hugetlbfs

The sibling mechanism is hugetlbfs: an explicitly pre-reserved pool of huge pages (e.g. 2 MiB or 1 GiB) that a process maps deterministically. Choose hugetlbfs when you need guaranteed huge pages with no allocation-time stalls and are willing to reserve memory up front (databases with a fixed buffer pool, DPDK, VMs with pinned RAM); choose THP when you want huge-page benefit opportunistically with zero application changes and can tolerate best-effort behavior. The two coexist on the same kernel without interference. See Huge Pages Overview for the decision framework and the 1 GiB gigantic-page story (only hugetlbfs offers 1 GiB; THP tops out at the PMD size).

See Also

Graph View

Backlinks