Two-Dimensional Paging (EPT and NPT)

Two-dimensional paging (TDP) is the hardware mechanism that lets a guest run its own page tables and lets the hypervisor independently control where guest memory actually lives — by making the CPU’s Memory-Management Unit (MMU) walk two page-table hierarchies on every Translation-Lookaside-Buffer (TLB) miss instead of one. The guest’s own tables map a guest-virtual address (GVA) to a guest-physical address (GPA), exactly as on bare metal; a second, hypervisor-owned table — Intel’s Extended Page Tables (EPT) or AMD’s Nested Page Tables (NPT), marketed for a time as Rapid Virtualization Indexing (RVI) — then maps that GPA to the real host-physical address (HPA). The MMU resolves GVA → GPA → HPA in a single nested traversal, so the guest never traps when it edits its own page tables (KVM mmu.rst, v6.12). This replaced the older software technique of Shadow Page Tables, trading a cheaper page-table update for a more expensive page walk — up to 24 memory accesses on a full miss with four-level paging on both sides, a figure that comes from the architecture-research literature and not from either vendor manual (Bhargava, Serebrin, Spadini & Manne, ASPLOS 2008, who give the general form nm + n + m; restated by Gandhi, Hill & Swift, ISCA 2016). Crucially, the EPT/NPT layer sits on top of the host’s own memory management, so guest pages remain ordinary host pages — swappable, dedup-able with KSM, and backed by transparent huge pages.

Version, editions, and source hygiene

Kernel source is pinned to Linux 6.12, a maintained long-term-support release (mainline is on the 7.x series as of 2026-09-04; the machine these notes were prepared on runs 7.1.8). Vendor architecture text is quoted from two manuals that were downloaded and verified against their own cover pages before citing:

• Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3 (3A, 3B, 3C & 3D): System Programming Guide, order number 325384-092US, June 2026, 1,602 pages. Note that the EPT chapter has moved: it is now Chapter 31, “VMX Support for Address Translation”, not Chapter 28 or 29 as older write-ups (and older editions) say.
• AMD64 Architecture Programmer’s Manual, Volume 2: System Programming, publication 24593, revision 3.45, July 2026, 875 pages.

That verification step was not ceremonial. The obvious-looking AMD URL www.amd.com/…/programmer-references/24593.pdf returns HTTP 404 with a 150 KB HTML error page served under a .pdf filename — a source that looks fetched to a status check and to a casual filename check. The real document is reachable only through the docs.amd.com document-service API. Likewise, the Wayback Machine’s “closest” snapshot of VMware’s RVI performance PDF is a 17 KB HTML error page under the .pdf URL; only snapshots on or before 2015-02-26 are the real document.

Mental Model

The cleanest way to think about TDP is two completely separate page-table owners that the silicon stitches together at walk time. The guest operating system owns the first table; it believes it is running on bare metal and manipulates its CR3 register and page tables freely. The hypervisor owns the second table (the EPT/NPT structure) and registers its root with the CPU through a dedicated pointer — the EPT Pointer (EPTP) on Intel or the nested CR3 (nCR3) in the guest’s Virtual Machine Control Block on AMD. Neither owner can see the other’s table; the MMU is the only entity that reads both.

The defining property is non-interference: when the guest writes its own page tables, nothing traps, because those tables only produce GPAs, which are still “fake” addresses the hardware will translate again. Contrast this with Shadow Page Tables, where the hypervisor had to intercept every guest page-table edit to keep a merged GVA → HPA table consistent. TDP eliminates that interception entirely at the cost of a longer walk.

flowchart LR
  subgraph GUEST["Guest-owned (no traps on edit)"]
    GVA["GVA<br/>(guest virtual)"]
    GPT["Guest page tables<br/>(4 levels, guest CR3)"]
    GPA["GPA<br/>(guest physical)"]
    GVA --> GPT --> GPA
  end
  subgraph HOST["Hypervisor-owned (EPT / NPT)"]
    EPT["EPT / NPT tables<br/>(4 levels, EPTP / nCR3)"]
    HPA["HPA<br/>(host physical)"]
    GPA --> EPT --> HPA
  end
  HOST -.->|"sits on top of"| MM["Host MM: reclaim, KSM,<br/>THP, NUMA, swap"]

The two-dimensional translation. What it shows: the guest’s four-level walk produces a GPA, which is not a real address; the hypervisor’s EPT/NPT walk then translates that GPA to the real HPA. The guest never sees the second stage and never traps editing the first. The insight to take: the GPA is the seam between two independently-managed worlds — guest page tables above it, host memory management (swap, KSM, huge pages, NUMA) below it. A “guest physical page” is really just an entry in the EPT pointing at a host page the kernel is free to move.

Why a Second Table at All — The Problem TDP Solves

A guest cannot be given real host-physical addresses. If two guests each believed they owned physical page 0, they would collide; if a guest could name a host-physical address directly, it could read another tenant’s RAM. So the hypervisor invents a per-guest fiction — guest-physical address space — that looks like contiguous RAM starting at 0 but is actually a sparse, relocatable mapping onto scattered host pages (the memory-slot machinery defines which GPA ranges map to which host virtual-memory regions).

Before hardware help, KVM bridged this gap in software with Shadow Page Tables: it built a merged table mapping GVA → HPA directly, loaded that into the real CR3, and trapped every guest write to the guest’s own page tables (and every guest CR3 reload) to keep the merge consistent. The walk was native-fast — four memory accesses on x86-64 — but the maintenance was brutal: marking a page copy-on-write or flushing a TLB cost thousands of cycles in VM-exit traps (Gandhi et al. §II.B). TDP inverts that trade: zero maintenance traps, longer walks.

The two schemes are best understood as opposite answers to the same question — where do you pay?

flowchart LR
  subgraph SP["Shadow paging — pay on <b>update</b>"]
    direction TB
    S1["guest edits its page tables"] -->|"<b>TRAP</b><br/>write-protected"| S2["VM exit to hypervisor"]
    S2 --> S3["re-merge gva→hpa<br/>into the shadow table"]
    S3 --> S4["VM entry"]
    S5["guest reloads CR3<br/>(every context switch)"] -->|"<b>TRAP</b>"| S2
    S6["TLB miss"] --> S7["<b>4</b> memory accesses<br/>— native speed"]
  end
  subgraph TDP["Two-dimensional paging — pay on <b>miss</b>"]
    direction TB
    T1["guest edits its page tables"] -->|"no trap —<br/>it only writes gPAs"| T2["nothing happens"]
    T3["guest reloads CR3"] -->|"no trap"| T2
    T4["TLB miss"] --> T5["<b>24</b> memory accesses<br/>— 3.9-4.6x native walk latency"]
  end

The cost inversion at the heart of the design. What it shows: shadow paging keeps the native four-access walk but charges a VM exit — thousands of cycles — for every guest page-table write and every CR3 reload; two-dimensional paging charges nothing for those and instead makes each TLB miss up to six times more expensive. The insight to take: this is a workload-dependent trade, not a strict improvement. Processes forking and exec-ing constantly (page-table churn) love TDP; a single large process with a huge working set and stable mappings (TLB churn, no updates) is exactly where shadow paging’s cheap walk still wins — which is why VMware measured EPT as 13–15% slower on SPECjbb2005 with small pages.

The Two-Dimensional Walk, Step by Step

On a TLB hit, TDP costs nothing — the cached entry already maps GVA → HPA directly, tagged with the guest’s identity (see VPID/ASID below), so the translation is as fast as native (Gandhi et al. Table I). The cost appears only on a TLB miss, and it is the central performance fact of memory virtualization.

On bare metal, an x86-64 TLB miss requires the page-walker to read 4 page-table entries (one per level: PML4 → PDPT → PD → PT). Each read is itself a physical memory access, because page-table entries store physical addresses.

Under TDP, the guest’s page-table entries store guest-physical addresses, not real ones. So every physical access the guest walker would have made must itself be translated through the EPT/NPT table — which is its own four-level walk. The arithmetic, stated verbatim by Gandhi, Hill and Swift, is:

“the page table memory references grow from a native 4 to a virtualized 24 references: 4 access to translate gptr (since each gPA requires access to host page table) and each of the 4 levels of the guest page table (guest page table holds gPA) plus 4 references for the guest page table itself to obtain the final hPA: 4 × 5 + 4 references.” (ISCA 2016, §II.A)

Walking that symbol-by-symbol: there are 5 “rows” in the walk that each hold a guest-physical pointer needing translation — the guest CR3/page-table pointer (gptr) plus the four guest page-table levels. Four of these (gptr and the upper three guest levels) each cost a 5-access nested step (1 access to read the guest entry + 4 EPT accesses to translate the GPA it contains into an HPA), and the final guest page-table entry (the leaf PTE) plus the final data-page GPA costs the remaining 4 EPT accesses. The paper’s Table II makes the per-level breakdown explicit: the page-table pointer costs 4 (native 0), and each of the four guest levels costs 5 (native 1) — summing to 24 for nested paging versus 4 for native.

The 24 is worth deriving from first principles rather than memorising, and the original derivation is more general than the number. Bhargava, Serebrin, Spadini and Manne — the AMD architects who designed the hardware that makes two-dimensional paging bearable — state it as a formula in the paper that introduced the two-dimensional page-walk cache:

“Although nested paging removes the overhead of hypervisor intervention, it increases the maximum number of page entry references architecturally required to generate a system physical address. If a guest page walk has n levels and a nested page walk has m levels, a 2D walk requires nm + n + m page entry references. For example, a 2D page walk with four-level guest paging and four-level nested paging has six times more page entry references than a four-level native page walk.” (Bhargava et al., ASPLOS 2008, §1)

Walking nm + n + m symbol by symbol with n = m = 4: nm = 16 is the cost of translating the four guest-physical pointers the guest walker needs — the value in gCR3, and the next-level addresses it finds in the guest PML4E, PDPTE and PDE — each of which is a gPA requiring its own four-level EPT/NPT walk. n = 4 is the cost of finally reading each of the four guest page-table entries once its host address is known. m = 4 is the last EPT/NPT walk, translating the gPA of the data page itself. Sum: 16 + 4 + 4 = 24, against 4 on bare metal — the “six times more page entry references” of the quotation.

Here is the whole thing as a grid. The horizontal axis is the nested (EPT/NPT) dimension; the vertical axis is the guest dimension. This is why it is called two-dimensional paging:

Guest-dimension step	nested L4	nested L3	nested L2	nested L1	then read	running total
translate the gPA in gCR3	1	2	3	4	5 — guest PML4E	5
translate the gPA in the guest PML4E	6	7	8	9	10 — guest PDPTE	10
translate the gPA in the guest PDPTE	11	12	13	14	15 — guest PDE	15
translate the gPA in the guest PDE	16	17	18	19	20 — guest PTE	20
translate the data gPA from the guest PTE	21	22	23	24	(the datum itself)	24

The two-dimensional page walk, numbered exactly as in Bhargava et al. Figure 1(b). What it shows: four “rows” that each cost five accesses (four nested plus one guest read), and a fifth row of four nested accesses with no guest read after it, because the guest walk is already finished. The insight to take: every single guest page-table read is preceded by a complete four-level walk of a second table. The guest walker never gets to dereference a pointer directly — it only ever holds addresses that are themselves fictional.

flowchart LR
  GVA["GVA<br/>0x7f3a_1234_5678"] --> R1
  subgraph R1["Row 1 — gCR3 holds a gPA"]
    direction LR
    A1["nested walk<br/>4 accesses<br/>steps 1-4"] --> A2["read guest PML4E<br/>step 5"]
  end
  subgraph R2["Row 2 — PML4E holds a gPA"]
    direction LR
    B1["nested walk<br/>4 accesses<br/>steps 6-9"] --> B2["read guest PDPTE<br/>step 10"]
  end
  subgraph R3["Row 3 — PDPTE holds a gPA"]
    direction LR
    C1["nested walk<br/>4 accesses<br/>steps 11-14"] --> C2["read guest PDE<br/>step 15"]
  end
  subgraph R4["Row 4 — PDE holds a gPA"]
    direction LR
    D1["nested walk<br/>4 accesses<br/>steps 16-19"] --> D2["read guest PTE<br/>step 20"]
  end
  subgraph R5["Row 5 — the guest PTE holds the data gPA"]
    direction LR
    E1["nested walk<br/>4 accesses<br/>steps 21-24"] --> E2["HPA / sPA<br/>of the datum"]
  end
  R1 --> R2 --> R3 --> R4 --> R5
  E2 --> OUT["one TLB entry:<br/>GVA → HPA, tagged VPID/ASID<br/><b>24 memory accesses spent</b>"]

The same walk drawn as nested loops. What it shows: the outer loop runs once per guest paging level plus once for the data page; the inner loop is a complete four-level EPT/NPT walk each time. The insight to take: the product structure — n outer iterations each containing m inner accesses — is exactly why the cost is multiplicative (nm) rather than additive, and why shrinking either dimension (huge pages in the guest, huge pages in the EPT) helps disproportionately.

Attribution correction — neither vendor manual states "24"

An earlier revision of this note implied the 24-access figure could be confirmed against Intel SDM Vol. 3C and AMD APM Vol. 2. It cannot. Both manuals were read in full for this note and neither performs the arithmetic:

• Intel works the mechanism out longhand for a two-level guest and stops: “If CR0.PG = 1, the translation of a linear address to a physical address requires multiple translations of guest-physical addresses using EPT” (§31.3.1), then enumerates the three steps for a 32-bit non-PAE guest without ever multiplying out the four-level case.
• AMD says only “a TLB miss causes several nested table walks” (§15.25.5) and, in the fault-ordering rules, “Steps 1 and 2 are repeated for each level of the guest page table that is traversed” (§15.25.6) — the structural statement from which 24 falls out, but not the number.

The number is entirely from the architecture-research literature: Bhargava et al. (ASPLOS 2008) for the general nm + n + m formula, and Gandhi, Hill & Swift (ISCA 2016) for the 4 × 5 + 4 restatement quoted above. Cite it that way. The configuration assumptions still hold: four-level paging on both dimensions, a cold walk with no page-walk-cache hits, and no large pages.

Five-level paging changes the arithmetic in both dimensions, and the EPT side of it is architecturally explicit. Intel: “The EPT translation mechanism can be configured in either of two modes: 4-level EPT or 5-level EPT. 4-level EPT accesses at most 4 EPT paging-structure entries (an EPT page-walk length of 4) to translate a guest-physical address and uses only bits 47:0 of each guest-physical address. In contrast, 5-level EPT may access up to 5 EPT paging-structure entries (an EPT page-walk length of 5) and uses guest-physical address bits 56:0” (SDM Vol. 3C §31.3.2). The mode is selected by three bits of the EPT pointer: “bits 5:3 contain a value one less than EPT page-walk length. Thus, a value of 3 configures 4-level EPT, while a value of 4 configures 5-level EPT.” Linux encodes exactly that, and derives the level back out of the pointer:

#define VMX_EPTP_PWL_MASK   0x38ull
#define VMX_EPTP_PWL_4      0x18ull      /* 3 << 3 */
#define VMX_EPTP_PWL_5      0x20ull      /* 4 << 3 */
 
static inline u8 vmx_eptp_page_walk_level(u64 eptp)
{
	u64 encoded_level = eptp & VMX_EPTP_PWL_MASK;
	if (encoded_level == VMX_EPTP_PWL_5)
		return 5;
	WARN_ON_ONCE(encoded_level != VMX_EPTP_PWL_4);
	return 4;
}

(arch/x86/include/asm/vmx.h, v6.12)

With n = m = 5 the formula gives 25 + 5 + 5 = 35 accesses. AMD reaches the same place differently: there is no NPT-specific depth control at all, because “the extra translation uses the same paging mode as the VMM used when it executed the most recent VMRUN” (APM Vol. 2 §15.25.3) — the nested table simply inherits the host’s paging mode, so a host running with CR4.LA57=1 gets five-level NPT automatically.

In practice the full 24-access walk is rare. The MMU caches intermediate translations in page-walk caches and paging-structure caches, and the EPT itself can use huge pages (2 MiB or 1 GiB) that collapse levels — the kernel’s TDP MMU explicitly maps and splits these, e.g. it can “replace [a] page table with an equivalent 1GiB hugepage” (tdp_mmu.c, v6.12). Backing guest RAM with huge pages is the single most effective way to cut TDP walk cost, because it shrinks both dimensions of the walk simultaneously.

What Makes 24 Tolerable

If every TLB miss cost 24 memory accesses, hardware virtualisation would be unusable. Three mechanisms shrink the real cost, and they attack different parts of the nm + n + m product.

flowchart TD
  MISS["TLB miss on a GVA"] --> NTLB{"<b>Nested TLB</b><br/>gPA of a guest PTE<br/>already translated?"}
  NTLB -->|hit| SKIP["skip 4 nested accesses<br/>for this row"]
  NTLB -->|miss| PWC{"<b>2D page-walk cache</b><br/>upper-level entry cached?<br/>24-entry, fully assoc."}
  PWC -->|hit| CHEAP["1 access from a 2-cycle<br/>structure instead of<br/>an L2/L3/DRAM trip"]
  PWC -->|miss| MEM["real memory-hierarchy access<br/>L2 → L3 → DRAM<br/>~100 cycles on an L2 miss"]
  SKIP --> HUGE
  CHEAP --> HUGE
  MEM --> HUGE
  HUGE{"<b>Huge pages</b>"} -->|"2 MiB in guest<br/>n: 4 → 3"| G["one fewer guest row<br/>= 5 fewer accesses"]
  HUGE -->|"2 MiB in EPT/NPT<br/>m: 4 → 3"| H["one fewer nested access<br/>in <i>every</i> row = 5 fewer"]
  HUGE -->|"1 GiB both sides<br/>n = m = 2"| I["nm+n+m = 4+2+2 = <b>8</b><br/>vs 24"]
  G --> OUT["fill one combined TLB entry"]
  H --> OUT
  I --> OUT

The three defences against the 24-access worst case. What it shows: the Nested TLB removes whole rows, the page-walk cache makes surviving accesses cheap rather than removing them, and huge pages shrink both n and m in the formula. The insight to take: huge pages are the only lever that reduces the architectural access count rather than the latency of each access, and because the cost is nm + n + m, shrinking both dimensions from 4 to 2 takes 24 down to 8 — a far bigger win than the arithmetic first suggests.

Page-walk caches are the oldest of the three and predate virtualisation. Bhargava et al. describe the one-dimensional original: “The AMD Opteron processor further benefits from the frequent reuse of page entry references by accelerating native page walks with a page walk cache (PWC). The PWC is a small, fast, fully-associative, physically-tagged page entry cache. A PWC hit prevents a page entry reference from accessing the memory hierarchy. The PWC stores page entries from all page table levels except L1, which is effectively stored in the TLB.” Their contribution was extending it into the second dimension: “The 2D PWC design … stores data for all 24 page table references of the 2D page walk, turning the 20 unconditional cache hierarchy accesses of 1D PWC into 16 likely PWC hits … and four possible PWC hits” (ASPLOS 2008, §4.1–4.2). Their modelled PWC is 24 entries, fully associative, LRU, 2-cycle access, against an 11-cycle PWC-miss-to-L2-hit and roughly 100 cycles for an L2 miss.

The Nested TLB is the virtualisation-specific addition, and its design rationale is worth reading because it explains which translations are worth caching: “The primary goal of the NTLB is to reduce the average number of page entry references that take place during a 2D page walk. … The NTLB uses the guest physical address of the guest page entry to cache the corresponding nL1 entry. Caching nL1 page entries is preferable to caching G entries because of the superior reuse characteristics of nL1 … In addition, storing nL1 page entries allows one NTLB entry to exploit spatial locality and provide translations for all page entries that reside in the same page of memory.” On a hit, “nested references 1-4 … [are] skipped” — an entire row of the grid above disappears. The authors are honest about the cost: “note that accessing the new NTLB structure imposes latency in the page walk unrelated to the PWC that is not present in the other schemes.”

Huge pages are the lever the operator actually controls, and they work on both dimensions independently. KVM’s TDP MMU maps at the largest level the host memory allows (kvm_mmu_hugepage_adjust() inside kvm_tdp_mmu_map), and can promote a fully-populated lower level, replacing “[a] page table with an equivalent 1GiB hugepage” (tdp_mmu.c, v6.12). The hardware advertises which sizes the EPT supports through capability bits Linux names directly — VMX_EPT_2MB_PAGE_BIT (bit 16) and VMX_EPT_1GB_PAGE_BIT (bit 17) of IA32_VMX_EPT_VPID_CAP.

There is a matching failure mode when the two dimensions disagree, which both AMD and the ASPLOS paper call page splintering. AMD: “When an address is mapped by guest and nested page table entries with different page sizes, the TLB entry that is created matches the size of the smaller page” (APM Vol. 2 §15.25.9). Bhargava puts the consequence in TLB-capacity terms: “a splintered 2MB page in the guest could require as many as 512 4KB TLB entries.” This is the mechanism behind the standard operational advice that a guest using transparent huge pages gains almost nothing if the host has backed its RAM with 4 KiB pages — the host silently splinters every one of them.

Inside an EPT Entry — and Why AMD’s Looks Nothing Like It

The single largest architectural difference between the two vendors’ implementations is not performance; it is that Intel invented a new page-table entry format and AMD reused the existing one.

Intel’s EPT entry has no “present” bit, no user/supervisor bit, and no NX bit. Instead it carries three independent permission bits at the bottom, and presence is derived from them: “An EPT paging-structure entry is present if any of bits 2:0 is 1; otherwise, the entry is not present” (SDM Vol. 3C §31.3.2). The leaf format, transcribed from Table 31-7, “Format of an EPT Page-Table Entry that Maps a 4-KByte Page”:

packet-beta
0: "R"
1: "W"
2: "X"
3-5: "memtype"
6: "IgnPAT"
7: "Ign"
8: "A"
9: "D"
10: "Xu"
11: "Ign"
12-51: "Physical address of the 4-KByte page (bits M-1:12; bits 51:M reserved, must be 0)"
52-56: "Ignored"
57: "VerifyGuestPaging"
58: "PagingWrite"
59: "Ign"
60: "SupShadowStack"
61: "SubPageWrite"
62: "Ign"
63: "SuppressVE"

The complete 64-bit EPT PTE that maps a 4 KiB page, transcribed bit-for-bit from SDM Vol. 3C Table 31-7. What it shows: the permission triple at bits 2:0, the memory type embedded in the entry itself at bits 5:3 with an ignore-PAT override at 6, the accessed/dirty pair at 8/9, the user-mode execute bit at 10 that exists only under mode-based execute control, and five feature bits in the top byte that are each gated on a separate VM-execution control. The insight to take: bits 0–2 do the work that P, R/W and NX do in an ordinary x86 PTE, but as three independent enables rather than a present bit plus modifiers — which is precisely what makes illegal combinations expressible, and hence what makes “EPT misconfiguration” a distinct fault from “EPT violation”. Bit 63 is why Linux’s shadow_present_mask for EPT is READABLE | VMX_EPT_SUPPRESS_VE_BIT and not just the read bit: KVM must set suppress-#VE on every present entry so that a write or execute violation produces a clean VM exit rather than being convertible into a guest-visible virtualisation exception.

The non-leaf entries are a different shape entirely, which is the first trap for anyone transcribing these tables:

packet-beta
0: "R"
1: "W"
2: "X"
3-7: "Reserved, must be 0"
8: "A"
9: "Ign"
10: "Xu"
11: "Ign"
12-51: "Physical address of the next EPT paging structure (bits M-1:12; 51:M reserved)"
52-63: "Ignored"

An EPT PML4E — representative of every non-leaf entry (SDM Tables 31-1, 31-2, 31-4, 31-6). What it shows: bits 7:3 are “Reserved (must be 0)” rather than memory type and page-size, bit 9 is Ignored rather than dirty, and the entire top twelve bits are Ignored — none of the leaf-only feature bits exist here. The insight to take: only bits 0, 1, 2, 10, 8 and the address field are common to all seven EPT table formats. A diagram or a parser that assumes one uniform EPT entry layout is wrong at four of the seven levels, and bit 7 alone means three different things depending on which table you are in.

The second trap is bit 7, which means three different things depending on the table you are reading. In a 4 KiB PTE (Table 31-7) it is simply “Ignored”. In a PDE that references an EPT page table (Table 31-6) it is “Must be 0 (otherwise, this entry maps a 2-MByte page)”. In a PDE that maps a 2 MiB page (Table 31-5) it is “Must be 1 (otherwise, this entry references an EPT page table)”, and the same inversion applies to a PDPTE mapping a 1 GiB page (Table 31-3). A third asymmetry: bit 61, sub-page write permissions, exists only in the 4 KiB PTE — Tables 31-3 and 31-5 list bits 62:61 as “Ignored”, so sub-page write protection cannot be applied to a large page.

AMD needed none of this. Grepping the full 875-page APM Volume 2 turns up no table defining a nested-page-table-entry format at all, because there is nothing new to define — nested page tables are ordinary long-mode x86-64 page tables. The manual refers to their bits by the standard names throughout: “the U/S bit in the nested page table” and “the NX bit” in the guest-mode-execute-trap section, “The PCD/PWT/PATi bits in the nested and guest page table entries” in the memory-typing section, and “the dirty and accessed bits are always set in the nested page table entries that were touched during nested page table walks” in §15.25.5. The nested fault’s error code is likewise the ordinary page-fault code: “EXITINFO1 delivers an error code similar to a PF error code” with bits P, RW, US, RSV, ID, SS (§15.25.6).

	Intel EPT	AMD NPT
Root pointer	EPTP, a VMCS field (SDM Table 27-9)	nCR3, from the VMCB N_CR3 field, loaded by VMRUN and not saved back on #VMEXIT
Enable control	“enable EPT” VM-execution control	NP_ENABLE bit in the VMCB; CPUID Fn8000_000A_EDX[NP]
Depth	4 or 5 levels, chosen by EPTP[5:3] = walk_length − 1	Whatever paging mode the host was in at VMRUN — inherited, not independently configurable
Entry format	Bespoke. Seven distinct table layouts (SDM Tables 31-1…31-7)	Identical to an ordinary x86-64 PTE. No new format exists
“Present”	Derived: present iff any of bits 2:0 is set (or bit 10 with mode-based execute control)	The standard P bit
Read / write / execute	Independent bits 0, 1, 2 (+ bit 10 for user-mode execute under MBEC)	Standard P + R/W + NX, with U/S repurposed for guest-mode execute trap (GMET)
Memory type	In the leaf entry, bits 5:3, plus ignore-PAT at bit 6. “The MTRRs have no effect on the memory type used for an access to a guest-physical address” (§31.3.7.2)	A two-stage combining lattice: guest PAT × host PAT (APM Table 15-19) then the result × MTRR (Table 15-20), with MTRRs still live on the system-physical address
Distinct fault kinds	EPT violation and EPT misconfiguration — two separate VM exit reasons	One exit, #VMEXIT(NPF); EXITINFO1 bit 32 says the fault was on the final gPA, bit 33 that it was while walking the guest page tables
TLB tag	VPID (plus PCID and the EPT root address)	ASID (VMCB offset 058h, bits 31:0); ASID 0 reserved for the host
Invalidation	INVEPT, INVVPID	TLB_CONTROL byte in the VMCB, plus INVLPGA
Confidential-computing extension	TDX / SEAM, with a separate Secure EPT owned by the TDX module	SEV-SNP, with the Reverse Map Table checked after the NPT

AMD’s memory-typing design deserves its own note, because it is where the two architectures diverge most sharply in behaviour rather than encoding. Intel bakes a memory type into the EPT leaf and switches the MTRRs off for guest-physical accesses. AMD combines: “When nested paging is enabled, the processor combines guest and nested page table memory types”, first guest-PAT against host-PAT (Table 15-19) and then the combined type against the MTRR type looked up on the system physical address (Table 15-20). Combining two types that disagree about cacheability creates a coherency hole, and AMD closes it by inventing a type: “A new memory type WC+ is introduced. WC+ is an uncacheable memory type, and combines writes in write-combining buffers like WC. Unlike WC (but like the CD memory type), accesses to WC+ memory also snoop the caches on all processors (including self-snooping the caches of the processor issuing the request) to maintain coherency. … When combining nested and guest memory types that are incompatible with respect to caching, the WC+ memory type is used instead of WC” (APM Vol. 2 §15.25.8). The manual also notes the consequence for emulation: “there is no hardware support for guest MTRRs; the VMM can simulate their effect by altering the memory types in the nested page tables.”

Linux’s KVM encodes both models in one place, which is a compact way to see the difference. kvm_mmu_set_ept_masks() re-points every generic shadow_* mask at the EPT encoding:

void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only)
{
	shadow_user_mask	= VMX_EPT_READABLE_MASK;      /* EPT has no U/S bit */
	shadow_accessed_mask	= has_ad_bits ? VMX_EPT_ACCESS_BIT : 0ull;
	shadow_dirty_mask	= has_ad_bits ? VMX_EPT_DIRTY_BIT  : 0ull;
	shadow_nx_mask		= 0ull;                        /* no NX bit either */
	shadow_x_mask		= VMX_EPT_EXECUTABLE_MASK;     /* a positive X bit instead */
	/* VMX_EPT_SUPPRESS_VE_BIT is needed for W or X violation. */
	shadow_present_mask	=
		(has_exec_only ? 0ull : VMX_EPT_READABLE_MASK) | VMX_EPT_SUPPRESS_VE_BIT;
	shadow_memtype_mask	= VMX_EPT_MT_MASK | VMX_EPT_IPAT_BIT;
	...
}

(arch/x86/kvm/mmu/spte.c, v6.12)

Line by line, this is the difference table above rendered as code. shadow_user_mask — the “this page is user-accessible” concept — maps onto EPT’s read bit, because EPT has no user/supervisor distinction. shadow_nx_mask becomes zero and shadow_x_mask becomes a real bit, because EPT expresses executability positively where x86 expresses non-executability negatively. shadow_present_mask is the read bit unless the CPU supports execute-only translations, in which case there is no bit that must be set for presence — plus the suppress-#VE bit on every present entry. And shadow_memtype_mask exists at all only under EPT; the NPT path sets shadow_memtype_mask = 0 with the comment “For shadow paging and NPT, KVM uses PAT entry ‘0’ to encode WB memtype in the SPTEs, i.e. relies on host MTRRs to provide the correct memtype (WB is the ‘weakest’ memtype).”

TLB Tagging — Why Transitions Don’t Flush

Without help, every VM-entry and VM-exit would have to flush the TLB, because a cached GVA → HPA entry for the guest is meaningless to the host and vice-versa. That flush would make virtualization transitions ruinously expensive. Both vendors solve this by tagging every TLB entry with an identifier so host and guest (and different guests) coexist in the same TLB:

• Intel — Virtual-Processor Identifier (VPID). Added with Nehalem (2008), VPID tags each cached linear-address translation with a per-vCPU identifier, so “no TLB flushes occur for VM-entries or VM-exits.” KVM enables it by default — enable_vpid = 1 (vmx.c, v6.12) — and flushes selectively with the INVVPID instruction (vpid_sync_vcpu_single, vpid_sync_vcpu_global in vmx_flush_tlb_all).
• AMD — Address-Space Identifier (ASID). AMD’s SVM tags each TLB entry with an ASID identifying the VM, distinguishing host from guest entries so a host↔guest switch need not wipe the TLB. A terminology note worth having: “Rapid Virtualization Indexing” and “RVI” do not appear anywhere in AMD’s architecture manual. A case-sensitive search of all 875 pages of APM Vol. 2 rev. 3.45 returns zero hits for either term; the ISA documentation calls the feature only “nested paging” (and, for the confidential-computing variant, “Secure Nested Paging”). RVI was a marketing name — VMware’s own evaluation paper was titled “Performance of Rapid Virtualization Indexing (RVI)” in 2008 and retitled the following year — so treat it as a product label, not an architectural term.

Ulrich Drepper described the tagging arrangement in 2007, before either implementation was widely deployed, and his framing is still the clearest one-sentence statement of what is actually cached: “The results of the additional address translation steps are also stored in the TLB. That means the TLB does not store the virtual physical address but, instead, the complete result of the lookup.” He also records how small the first version was: “AMD’s Pacifica extension introduced the ASID to avoid TLB flushes on each entry. The number of bits for the ASID is one in the initial release of the processor extensions; this is just enough to differentiate VMM and guest OS. Intel has virtual processor IDs (VPIDs) which serve the same purpose, only there are more of them. But the VPID is fixed for each guest domain and therefore it cannot be used to mark separate processes and avoid TLB flushes at that level, too.” (Drepper, Memory part 3: Virtual Memory, LWN, 9 October 2007). That last observation has since been overtaken: modern x86 pairs the VPID with the guest’s own PCID, so process switches inside a guest need not flush either — the SDM’s cache taxonomy above tags linear and combined mappings by VPID and PCID.

These are distinct from EPT/NPT: VPID/ASID tag the combined GVA → HPA result; EPT/NPT define how that result is computed on a miss. A useful mental split: VPID/ASID make the hit path cheap across transitions, EPT/NPT make the miss path correct without traps.

Intel’s manual states the motivation in one sentence and then defines three categories of cached information, which is the part most summaries omit and the part that explains why there are two distinct invalidation instructions:

“The original architecture for VMX operation required VMX transitions to flush the TLBs and paging-structure caches. This ensured that translations cached for the old linear-address space would not be used after the transition. Virtual-processor identifiers (VPIDs) introduce to VMX operation a facility by which a logical processor may cache information for multiple linear-address spaces. When VPIDs are used, VMX transitions may retain cached information and the logical processor switches to a different linear-address space.” (SDM Vol. 3C §31.1)

flowchart TB
  subgraph CACHES["What the logical processor may cache (SDM §31.4.1)"]
    direction TB
    LIN["<b>Linear mappings</b><br/>linear page number → page frame<br/>tagged by VPID + PCID<br/><i>'do not contain information from<br/>any EPT paging structure'</i>"]
    GP["<b>Guest-physical mappings</b><br/>gPA → page frame<br/>tagged by the EPT root address<br/><i>privileges derived from<br/>EPT paging structures</i>"]
    COMB["<b>Combined mappings</b><br/>linear page number → page frame<br/>tagged by VPID + PCID + EPT root<br/><i>'derived from <b>both</b> guest paging<br/>structures and EPT paging structures'</i>"]
  end
  INVVPID["<b>INVVPID</b><br/>types 0-3<br/>individual-addr / single-ctx /<br/>all-ctx / single-ctx-retaining-globals"] --> LIN
  INVVPID --> COMB
  INVEPT["<b>INVEPT</b><br/>types 1-2<br/>single-context / all-context"] --> GP
  INVEPT --> COMB
  TRANS["VM entry / VM exit<br/>with 'enable VPID' = 1"] -.->|"invalidates<br/><b>nothing</b>"| CACHES

The three TLB-entry categories and which instruction clears each. What it shows: combined mappings sit in the intersection — they encode both dimensions of the walk, so either instruction can invalidate them, while linear-only and guest-physical-only entries each have exactly one owner. The insight to take: VPID and EPT are orthogonal tags on the same TLB, and the reason both instructions exist is that a change to the guest’s page tables and a change to the hypervisor’s EPT invalidate different, overlapping subsets of what is cached.

The manual is explicit that transitions no longer flush, which is the entire performance argument for VPID:

“VMX transitions are not required to invalidate any guest-physical mappings. If the ‘enable VPID’ VM-execution control is 1, VMX transitions are not required to invalidate any linear mappings or combined mappings. … The INVVPID instruction is not required to invalidate any guest-physical mappings. The INVEPT instruction is not required to invalidate any linear mappings.” (SDM Vol. 3C §31.4.3.2)

Conversely, when VPID is off: “If the ‘enable VPID’ VM-execution control is 0, VM entries and VM exits invalidate linear mappings and combined mappings associated with VPID 0000H (for all PCIDs)” (§31.4.3.1) — that is the pre-2008 behaviour, and it is what kvm-intel.vpid=0 restores. Two further details worth knowing. The instructions ignore each other’s tags entirely: “INVEPT invalidates all the specified mappings for the indicated EPTP(s) regardless of the VPID and PCID values with which those mappings may be associated”, and symmetrically for INVVPID (SDM Vol. 3C Ch. 33). And an EPT violation performs its own targeted invalidation as a side effect: “An EPT violation invalidates any guest-physical mappings (associated with the current EPT root address) that would be used to translate the guest-physical address that caused the EPT violation” (§31.4.3.1) — which is why KVM does not need to issue an explicit INVEPT after installing the missing entry on a demand-fault path.

Older Intel documentation used different names for the same three categories, and they still appear in secondary material; the SDM keeps the mapping in footnotes: “Earlier versions of this manual used the term ‘VPID-tagged’ to identify linear mappings … ‘EPTP-tagged’ to identify guest-physical mappings … ‘dual-tagged’ to identify combined mappings.”

AMD’s ASID differs from VPID in a way that is easy to miss and that the APM calls out directly, because the meaning of the tag changes depending on which paging scheme is in use:

“TLB entries are tagged with Address Space Identifier (ASID) bits to distinguish different guest virtual address spaces when shadow page tables are used, or different guest physical address spaces when nested page tables are used. … This allows switching to a new process in a guest under shadow paging (changing CR3 contents), or to a new guest under nested paging (changing nCR3 contents), without flushing the TLBs.” (APM Vol. 2 §15.16)

“The VMM can give each guest a different ASID, so that TLB entries from different guests can coexist in the TLB. The ASID value of zero is reserved for the host; if the VMM attempts to execute VMRUN with a guest ASID of zero, the result is VMEXIT(VMEXIT_INVALID). Note that because an ASID is associated with the guest’s physical address space, it is common across all of the guest’s virtual address spaces within a processor. This differs from shadow page tables where ASIDs tag individual guest virtual address spaces.” (APM Vol. 2 §15.25.1)

Invalidation on AMD is not an instruction family but a byte in the VMCB, at offset 058h bits 39:32, read (but not modified) by VMRUN:

TLB_CONTROL	Function definition (APM Vol. 2 Table 15-9, verbatim)
00h	Do not flush
01h	Flush entire TLB (Should be used only on legacy hardware.)
03h	Flush this guest’s TLB entries
07h	Flush this guest’s non-global TLB entries

Encodings 03h and 07h are the flush-by-ASID feature, optional and advertised by CPUID Fn8000_000A_EDX[FlushByAsid]. The APM gives the rule a hypervisor must follow, which is exactly the situation KVM hits when it write-protects guest RAM for live migration: “If a hypervisor modifies a nested page table by decreasing permission levels, clearing present bits, or changing address translations and intends to return to the same ASID, it should use either TLB command 011b or 001b.” There is also INVLPGA (“Invalidate Page, Alternate ASID”), which takes a linear address in rAX and an ASID in ECX. One last asymmetry with ordinary x86: “When running with SVM enabled, global page table entries (PTEs) are global only within an ASID, not across ASIDs.”

Linux enables both by default and syncs selectively. On Intel, bool __read_mostly enable_vpid = 1; with module_param_named(vpid, enable_vpid, bool, 0444), and flushes go through vpid_sync_vcpu_single() / vpid_sync_vcpu_global() inside vmx_flush_tlb_all (vmx.c, v6.12). The read-only permission bits 0444 on the module parameter are themselves informative — these are boot-time-only knobs, because flipping them on a running host would leave stale TLB state behind.

EPT Violations vs EPT Misconfigurations

When the EPT/NPT walk fails, the CPU raises one of two distinct VM exits, and KVM treats them very differently. The Intel VMX handlers in vmx.c (v6.12) make the distinction concrete:

An EPT violation is a permission or presence fault — the guest touched a GPA whose EPT entry is not present, or lacks the requested read/write/execute right. This is the normal, expected fault that populates guest memory lazily (demand paging of guest RAM) or implements write-tracking (for dirty logging). handle_ept_violation() reads the exit qualification, decodes which access bits were violated, and routes the fault into the MMU:

static int handle_ept_violation(struct kvm_vcpu *vcpu)
{
	unsigned long exit_qualification;
	gpa_t gpa;
	u64 error_code;
 
	exit_qualification = vmx_get_exit_qual(vcpu);
	...
	/* Is it a read fault? */
	error_code = (exit_qualification & EPT_VIOLATION_ACC_READ)
		     ? PFERR_USER_MASK : 0;
	/* Is it a write fault? */
	error_code |= (exit_qualification & EPT_VIOLATION_ACC_WRITE)
		      ? PFERR_WRITE_MASK : 0;
	/* Is it a fetch fault? */
	error_code |= (exit_qualification & EPT_VIOLATION_ACC_INSTR)
		      ? PFERR_FETCH_MASK : 0;
	...
	gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS);
	...
	return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0);
}

Line by line: vmx_get_exit_qual() reads the exit qualification, the VMCS field that encodes why the fault happened; the three EPT_VIOLATION_ACC_* bits say whether the offending access was a read, write, or instruction fetch, and each is folded into an error_code using the same PFERR_* bit layout the kernel uses for ordinary page faults; vmcs_read64(GUEST_PHYSICAL_ADDRESS) retrieves the faulting GPA (the CPU helpfully records it); finally kvm_mmu_page_fault() hands the GPA and decoded error code to the MMU, which walks its own struct kvm_mmu_page tables and installs the missing SPTE. The TDP MMU’s kvm_tdp_mmu_map() is documented as handling exactly this: “Handle a TDP page fault (NPT/EPT violation/misconfiguration) by installing” the mapping (tdp_mmu.c, v6.12).

An EPT misconfiguration is a malformed entry fault — the EPT entry itself has an illegal bit combination (reserved bits set, or a permission encoding the hardware forbids, such as write-without-read). KVM deliberately creates such “misconfigured” entries to trap MMIO: a GPA that maps to an emulated device has no real backing page, so KVM installs a special non-memory SPTE that triggers a misconfig on access. handle_ept_misconfig() routes it as a reserved-bit fault:

static int handle_ept_misconfig(struct kvm_vcpu *vcpu)
{
	gpa_t gpa;
	...
	gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS);
	if (!is_guest_mode(vcpu) &&
	    !kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
		trace_kvm_fast_mmio(gpa);
		return kvm_skip_emulated_instruction(vcpu);
	}
 
	return kvm_mmu_page_fault(vcpu, gpa, PFERR_RSVD_MASK, NULL, 0);
}

The key distinction in code: the violation handler passes the decoded R/W/X error code (it is real memory, populate it), while the misconfig handler passes PFERR_RSVD_MASK — a reserved-bit fault — because the entry was intentionally malformed to flag “this GPA is MMIO, not RAM.” The misconfig path can short-circuit into the fast-MMIO bus (KVM_FAST_MMIO_BUS) for zero-length notification writes, skipping full instruction emulation. The comment in the handler is explicit that “[a] nested guest cannot optimize MMIO vmexits, because we have an nGPA here instead of the required GPA” (vmx.c, v6.12).

Resolved against the SDM (2026-09-04)

The hardware definitions were read directly from Intel SDM Vol. 3C §31.3.3, “EPT-Induced VM Exits” (the chapter has moved from 28/29 to 31), and they confirm the reserved/illegal-encoding reading:

“An EPT misconfiguration occurs when, in the course of translating a guest-physical address, the logical processor encounters an EPT paging-structure entry that contains an unsupported value (see Section 31.3.3.1). An EPT violation occurs when there is no EPT misconfiguration but the EPT paging-structure entries disallow an access using the guest-physical address (see Section 31.3.3.2). A page-modification log-full event occurs when the logical processor determines a need to create a page-modification log entry and the current log is full (see Section 31.3.6).”

Note the third exit kind, which the original version of this note omitted: a page-modification log-full event, produced by the Page-Modification Logging feature that KVM enables by default (enable_pml = 1, module_param_named(pml, enable_pml, bool, 0444) in vmx.c, v6.12) and uses to make dirty-page tracking for live migration cheap.

§31.3.3.1 gives the exhaustive list of misconfiguration conditions, and it is worth having in full because it is the specification of “which bit patterns are illegal”:

“An EPT misconfiguration occurs if translation of a guest-physical address encounters an EPT paging-structure entry that meets any of the following conditions:

• Bit 0 of the entry is clear (indicating that data reads are not allowed) and any of the following hold:
  • Bit 1 is set (indicating that data writes are allowed).
  • The processor does not support execute-only translations and either of the following hold: Bit 2 is set (indicating that instruction fetches are allowed); the ‘mode-based execute control for EPT’ VM-execution control is 1 and bit 10 is set …
  • The ‘EPT paging-write control’ VM-execution control is 1, the entry maps a page, and bit 58 is set …
• The entry is present (see Section 31.3.2) and one of the following holds:
  • A reserved bit is set. This includes the setting of a bit in the range 51:12 in position MAXPHYADDR or above …
  • The entry is the last one used to translate a guest physical address (either an EPT PDE with bit 7 set to 1 or an EPT PTE) and the value of bits 5:3 (EPT memory type) is 2, 3, or 7 (these values are reserved).”

So there are exactly three families: write-without-read, execute-without-read on a CPU lacking execute-only support, and a reserved bit or a reserved memory type. The SDM adds a forward-compatibility warning that matters for anyone using misconfigurations deliberately: “EPT misconfigurations result when an EPT paging-structure entry is configured with settings reserved for future functionality. Software developers should be aware that such settings may be used in the future and that an EPT paging-structure entry that causes an EPT misconfiguration on one processor might not do so in the future.”

That is precisely the sharp edge KVM chooses to stand on, and the kernel says so in a one-line comment that quotes the rule back:

/*
 * EPT Misconfigurations are generated if the value of bits 2:0
 * of an EPT paging-structure entry is 110b (write/execute).
 */
kvm_mmu_set_mmio_spte_mask(VMX_EPT_MISCONFIG_WX_VALUE,
			   VMX_EPT_RWX_MASK | VMX_EPT_SUPPRESS_VE_BIT, 0);

(arch/x86/kvm/mmu/spte.c, v6.12; VMX_EPT_MISCONFIG_WX_VALUE is defined in vmx.h as VMX_EPT_WRITABLE_MASK | VMX_EPT_EXECUTABLE_MASK, with the comment “The mask to use to trigger an EPT Misconfiguration in order to track MMIO”)

110b — writable and executable but not readable — is the first family of the SDM’s list, deliberately constructed. KVM is exploiting an architecturally-illegal encoding as a signalling channel: “this guest-physical address is a device, not RAM.”

The corresponding §31.3.3.2 makes the violation side equally concrete, and one clause in it is the foundation of the entire two-dimensional walk:

“An EPT violation occurs in any of the following situations: Translation of the guest-physical address encounters an EPT paging-structure entry that is not present … The access is a data read and, for any byte to be read, bit 0 (read access) was clear in any of the EPT paging-structure entries used to translate the guest-physical address of the byte. Reads by the logical processor of guest paging structures to translate a linear address are considered to be data reads. … The access is a data write and, for any byte to be written, bit 1 (write access) was clear … Writes by the logical processor to guest paging structures to update accessed and dirty flags are considered to be data writes.”

Those two emphasised sentences are the formal statement of the grid drawn earlier: the CPU’s own page-walker is treated as a guest making ordinary reads and writes, so every one of its accesses is itself subject to EPT permission checks. AMD says the same thing operationally rather than definitionally: “Unless Read Only Guest Page Tables is enabled for a guest, table walks for guest page tables are always treated as user writes at the nested page table level. For this reason, the page must be writable by user at the nested page table level, or else a VMEXIT(NPF) is raised, and the dirty and accessed bits are always set in the nested page table entries that were touched during nested page table walks for guest page table entries” (APM Vol. 2 §15.25.5). Which is a trap: a hypervisor that write-protects a guest page containing page tables — for dirty logging, say — will fault on the walker’s own A/D update, not only on guest stores.

Finally, the priority rule: §31.3.3.3, “Prioritization of EPT Misconfigurations and EPT Violations” — misconfiguration wins. That matters because it guarantees KVM’s MMIO trick is unambiguous; an MMIO SPTE will always report as a misconfiguration, never as a violation, regardless of what access the guest attempted.

Putting the whole exit path together:

flowchart TD
  ACC["guest access to a GPA<br/>(or the CPU's own page-walker<br/>reading a guest PTE)"] --> WALK["EPT walk"]
  WALK --> MIS{"entry has an<br/>unsupported value?<br/>W-without-R · X-without-R ·<br/>reserved bit · memtype 2/3/7"}
  MIS -->|"yes — <b>higher priority</b>"| EXITM["VM exit reason 49<br/>EPT_MISCONFIG"]
  MIS -->|no| VIO{"entry not present,<br/>or R/W/X bit clear<br/>for this access?"}
  VIO -->|yes| EXITV["VM exit reason 48<br/>EPT_VIOLATION<br/>+ exit qualification"]
  VIO -->|no| OK["translation succeeds<br/>fill a combined TLB entry"]

  EXITM --> HM["handle_ept_misconfig()<br/>gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS)"]
  HM --> FAST{"non-nested and a<br/>zero-length write to<br/>KVM_FAST_MMIO_BUS?"}
  FAST -->|yes| SKIP["trace_kvm_fast_mmio(gpa)<br/>kvm_skip_emulated_instruction()<br/><i>no instruction emulation at all</i>"]
  FAST -->|no| RSVD["kvm_mmu_page_fault(gpa,<br/><b>PFERR_RSVD_MASK</b>)<br/>→ MMIO emulation"]

  EXITV --> HV["handle_ept_violation()<br/>decode EPT_VIOLATION_ACC_READ/WRITE/INSTR<br/>into PFERR_USER/WRITE/FETCH_MASK"]
  HV --> MMU["kvm_mmu_page_fault(gpa, error_code)"]
  MMU --> TDP["kvm_tdp_mmu_map():<br/>'Handle a TDP page fault<br/>(NPT/EPT violation/misconfiguration)<br/>by installing page tables and SPTEs'"]
  TDP --> INS["install the missing SPTE<br/>at the largest level the host allows"]

The full exit path for both fault kinds. What it shows: two hardware exit reasons converging on one MMU entry point, distinguished only by the error code KVM synthesises — decoded R/W/X bits for a violation, the single PFERR_RSVD_MASK bit for a misconfiguration — plus one short-circuit that skips instruction emulation entirely for zero-length MMIO notification writes. The insight to take: PFERR_RSVD_MASK is the seam. KVM reuses the ordinary x86 page-fault error-code vocabulary for both, so the same kvm_mmu_page_fault handles a guest RAM demand-fault and a device access, and the “reserved bit” flag is what tells them apart.

TDP Sits On Top of Host Paging

The most consequential property for the rest of the kernel is that the EPT/NPT entry’s output — the HPA — is just a host page frame, and the host memory manager owns it. Guest “physical” RAM is, from the host’s point of view, ordinary anonymous memory in the VMM process’s address space (see Anonymous vs File-Backed Memory). That means:

• Swapping. The host can reclaim a guest page to swap; KVM tears down the EPT entry, and the next guest access raises an EPT violation that re-faults the page in. The guest never knows.
• Kernel Samepage Merging. Identical guest pages across VMs can be merged to one read-only host page; a guest write triggers an EPT violation → copy-on-write.
• Transparent Huge Pages and NUMA. The host can back guest RAM with 2 MiB/1 GiB pages and migrate it between NUMA nodes; the TDP MMU adjusts its mapping level accordingly (kvm_mmu_hugepage_adjust() in kvm_tdp_mmu_map).

This is the seam where memory virtualization meets the core MM subsystem, and where the subtle bugs live — e.g. an EPT entry must be invalidated before the host page is reused, or the guest reads stale data. KVM hooks the MM notifier chain (mmu_notifier) so that any host-side page movement zaps the corresponding SPTEs.

sequenceDiagram
    autonumber
    participant HOST as Host MM (reclaim / KSM / THP / NUMA / compaction)
    participant MN as mmu_notifier chain
    participant KVM as KVM MMU
    participant HW as EPT / NPT + TLB
    participant G as Guest vCPU

    Note over HOST: host decides to move, merge,<br/>or swap out a page backing guest RAM
    HOST->>MN: invalidate_range_start(mm, start, end)
    MN->>KVM: kvm_mmu_notifier_invalidate_range_start()
    KVM->>HW: zap the SPTEs covering those GPAs
    KVM->>HW: remote TLB flush (INVEPT / TLB_CONTROL)
    Note over KVM,HW: only now is it safe for the host<br/>to reuse the page frame
    HOST->>HOST: actually move / free / swap the page
    HOST->>MN: invalidate_range_end()
    G->>HW: guest touches the GPA again
    HW-->>G: EPT violation (Intel) or VMEXIT-NPF (AMD)
    G->>KVM: kvm_mmu_page_fault(gpa, error_code)
    KVM->>HOST: get_user_pages / fault the page back in
    KVM->>HW: install a fresh SPTE at the largest allowed level
    Note over G: the guest never observes any of this

How EPT/NPT composes with the host memory manager. What it shows: the mmu_notifier chain is the ordering guarantee — KVM must tear down and flush its second-level mappings before the host reuses a page frame, or the guest reads another tenant’s data. The insight to take: this ordering is the whole reason guest memory can be swapped, deduplicated, compacted and NUMA-migrated at all. Two-dimensional paging is not merely “a second table”; it is the hook that makes guest RAM behave like ordinary host anonymous memory, at the cost of an EPT violation on the next touch.

The TDP MMU — KVM’s Default Implementation

KVM has two MMU implementations. The legacy shadow MMU code (mmu.c) handles both true Shadow Page Tables (when hardware TDP is unavailable) and TDP, using a unified page-table data structure. Alongside it there is a separate, scalable TDP MMU (tdp_mmu.c) used only when hardware nested paging is present; it takes kvm->mmu_lock in read mode for the common fault path and uses RCU for page-table teardown, instead of serialising every fault on one write-held lock — which is what matters for VMs with hundreds of vCPUs faulting concurrently. Its history is datable precisely by existence-checking the file across release tags: arch/x86/kvm/mmu/tdp_mmu.c returns HTTP 404 at v5.9 and 200 at v5.10, so it was merged for Linux 5.10; it declared tdp_mmu_enabled = false through v5.14 and = true from v5.15 onward, so 5.15 is the release in which it became the default. As of v6.12 the flag has also moved into mmu.c and become boot-time-only (0444 rather than the original 0644): bool __read_mostly tdp_mmu_enabled = true; with a read-only module parameter module_param_named(tdp_mmu, tdp_mmu_enabled, bool, 0444) (mmu.c, v6.12).

Hardware support itself is exposed through two module flags, both defaulting on:

• Intel: tdp_enabled is set when EPT is available.
• AMD: bool npt_enabled = true; module_param_named(npt, npt_enabled, bool, 0444); (svm.c, v6.12).

The KVM documentation describes the resulting translation chains compactly: for a non-nested guest with TDP, the MMU resolves (gva->)gpa->hpa; for a nested guest (a hypervisor inside a guest, see Nested Virtualization) it resolves (ngva->)ngpa->gpa->hpa (mmu.rst, v6.12). The doc also explains the direct vs shadow distinction: “When the number of required translations matches the hardware, the mmu operates in direct mode; otherwise it operates in shadow mode” — TDP enables direct mode.

timeline
    title KVM's MMU implementations for two-dimensional paging
    2007 : KVM merged in Linux 2.6.20 — software shadow paging only
    2008 : EPT (Nehalem) and NPT (Barcelona) ship; KVM gains hardware TDP
         : VPID arrives with Nehalem, removing the flush on every transition
    2020 : Linux 5.10 — the TDP MMU is merged (tdp_mmu.c first appears), default OFF
    2021 : Linux 5.15 — tdp_mmu_enabled flips to true; the TDP MMU becomes the default
    2024 : Linux 6.12 (LTS) — flag moves to mmu.c, becomes read-only 0444
    2025 : Linux 6.16 — 'mirror' roots appear in tdp_mmu.c for TDX Secure EPT

The implementation timeline, with each date established by reading the file at that release tag rather than from memory. What it shows: two-dimensional paging arrived in hardware in 2008, but it took KVM twelve more years to grow an MMU designed around it rather than an adaptation of the shadow-paging code. The insight to take: the shadow MMU is still present in v6.12 and still handles TDP for the nested case; the TDP MMU is not a replacement so much as a fast path for the one configuration that matters most.

The two MMUs coexist rather than one having replaced the other, and the KVM documentation is explicit that a single VM can hold pages from both: “tdp_mmu_page: Is 1 if the shadow page is a TDP MMU page. This variable is used to separate TDP MMU page from the rest, since a VM may contain pages from both TDP MMU and shadow MMU” (mmu.rst, v6.12). The division of labour is by number of translations, and the doc states the rule that governs it: “The primary challenge is to encode between 1 and 3 translations into hardware that support only 1 (traditional) and 2 (tdp) translations. When the number of required translations matches the hardware, the mmu operates in direct mode; otherwise it operates in shadow mode.”

Configuration	Translations required	Hardware provides	Mode	Which KVM MMU
Guest, paging off, TDP	gpa→hpa (1)	2	direct	TDP MMU
Guest, paging on, TDP	gva→gpa→hpa (2)	2	direct	TDP MMU
Guest, paging on, no TDP	gva→hpa (needs 2, hardware has 1)	1	shadow	shadow MMU
Nested guest, L1 uses EPT	ngva→ngpa→gpa→hpa (3)	2	shadow	shadow MMU (guest_mmu)

Nested Virtualization and Shadow EPT

The fourth row of that table is where two-dimensional paging runs out of dimensions. A nested guest (L2) running under a hypervisor (L1) that is itself running under KVM (L0) needs ngva → ngpa → gpa → hpa — three translations — and the silicon offers two. Something must be collapsed, and the thing that gets collapsed is L1’s EPT.

flowchart TB
  subgraph WANT["What is logically required — 3 translations"]
    direction TB
    NGVA["ngva<br/>L2 virtual"] -->|"L2's own page tables<br/>(L2 owns, no traps)"| NGPA["ngpa<br/>L2 physical"]
    NGPA -->|"<b>L1's EPT</b><br/>(L1 owns, in L1's memory)"| GPA["gpa<br/>L1 physical"]
    GPA -->|"<b>L0's EPT</b><br/>(KVM owns)"| HPA["hpa<br/>real memory"]
  end
  subgraph HAVE["What the hardware can do — 2 translations"]
    direction TB
    NGVA2["ngva"] -->|"L2's page tables,<br/>loaded in guest CR3"| NGPA2["ngpa"]
    NGPA2 -->|"<b>shadow EPT</b><br/>a single merged table KVM builds<br/>and loads into the real EPTP"| HPA2["hpa"]
  end
  WANT -.->|"KVM merges L1's EPT<br/>with its own"| HAVE
  L1W["L1 writes its EPT"] -->|"<b>traps</b> — KVM must<br/>rebuild the merge"| HAVE

Nested EPT, also called shadow EPT. What it shows: KVM composes L1’s ngpa→gpa table with its own gpa→hpa table into one ngpa→hpa table and loads that into the hardware EPTP, so the CPU still sees only two dimensions. The insight to take: every objection to shadow paging returns at the nested level. L1’s EPT edits must now be trapped and the merge rebuilt, which is exactly the maintenance cost that hardware TDP was introduced to eliminate — nesting reintroduces it one level up.

KVM implements this by swapping in a second MMU context whose “guest page directory” is L1’s EPT pointer rather than a CR3:

static void nested_ept_init_mmu_context(struct kvm_vcpu *vcpu)
{
	WARN_ON(mmu_is_nested(vcpu));
 
	vcpu->arch.mmu = &vcpu->arch.guest_mmu;
	nested_ept_new_eptp(vcpu);
	vcpu->arch.mmu->get_guest_pgd     = nested_ept_get_eptp;
	vcpu->arch.mmu->inject_page_fault = nested_ept_inject_page_fault;
	vcpu->arch.mmu->get_pdptr         = kvm_pdptr_read;
 
	vcpu->arch.walk_mmu               = &vcpu->arch.nested_mmu;
}

(arch/x86/kvm/vmx/nested.c, v6.12)

Line by line: vcpu->arch.mmu — the MMU that programs the hardware — switches from root_mmu to guest_mmu, a separate context whose page tables shadow L1’s EPT. get_guest_pgd is repointed at nested_ept_get_eptp, so when the generic MMU code asks “where is the root of the table you are shadowing?” it gets L1’s EPTP from vmcs12 instead of a CR3. inject_page_fault becomes nested_ept_inject_page_fault, which synthesises an EPT violation VM exit into L1 rather than a #PF into the guest — because from L1’s point of view, its EPT is what faulted. And walk_mmu — the context used for software address translation during instruction emulation — becomes nested_mmu, a third context that knows the full three-level chain. Three kvm_mmu structures for one vCPU is the price of a translation the hardware cannot express.

The performance consequence is the reason nested virtualisation has a reputation. A page fault in L2 that must be resolved by L1 costs an exit to L0, a synthesised exit into L1, L1’s handling, a VMRESUME trapped by L0, and a rebuild of the merged table — and the underlying walk, when it does happen in hardware, is the same 24-access two-dimensional walk with a deeper logical structure behind it. Everything in the “what makes 24 tolerable” section applies with more force here: huge pages on both L0 and L1, and memory pinning, are not tuning suggestions for nested workloads but requirements. See Nested Virtualization for the full L0/L1/L2 model.

The Security Dimension: EPT as a Policy Substrate

Because the EPT sits between the guest’s idea of physical memory and the machine’s, it has become the natural place to attach memory-security policy — and, less happily, the natural place for hardware bugs to be mitigated.

L1TF (L1 Terminal Fault, CVE-2018-3620/3646) is the sharpest case, because it is a vulnerability in the address-translation path that EPT was supposed to seal. The kernel documentation states the problem bluntly: the attack “allows an attack of SGX and also works from inside virtual machines because the speculation bypasses the extended page table (EPT) protection mechanism” (Documentation/admin-guide/hw-vuln/l1tf.rst, v6.12). On a terminal fault — a page-table entry marked not-present — affected CPUs speculatively forward the entry’s address bits to the L1 data cache before the fault is architecturally taken, leaking whatever line happens to be there. Under virtualisation, the address bits in a guest-controlled non-present entry are guest-controlled, so a malicious guest can point speculation at arbitrary host physical memory.

KVM’s structural mitigation is to make sure a non-present SPTE’s address field cannot name real memory, by relocating the frame number into bits above the physical address width:

	shadow_nonpresent_or_rsvd_mask = 0;
	low_phys_bits = boot_cpu_data.x86_phys_bits;
	if (boot_cpu_has_bug(X86_BUG_L1TF) &&
	    !WARN_ON_ONCE(boot_cpu_data.x86_cache_bits >=
			  52 - SHADOW_NONPRESENT_OR_RSVD_MASK_LEN)) {
		low_phys_bits = boot_cpu_data.x86_cache_bits
			- SHADOW_NONPRESENT_OR_RSVD_MASK_LEN;
		shadow_nonpresent_or_rsvd_mask =
			rsvd_bits(low_phys_bits, boot_cpu_data.x86_cache_bits - 1);
	}

(arch/x86/kvm/mmu/spte.c, v6.12)

Walking it: SHADOW_NONPRESENT_OR_RSVD_MASK_LEN is 5, so KVM reserves the top five bits of the cache-indexing address width. x86_cache_bits rather than x86_phys_bits is deliberate — the comment explains that “Some Intel CPUs address the L1 cache using more PA bits than are reported by CPUID. Use the PA width of the L1 cache when possible to achieve more effective mitigation.” make_mmio_spte then splits the guest frame number, storing its low part in place and its top five bits shifted up into that reserved region, so the value the speculative path reads as a physical address is guaranteed to be above MAXPHYADDR and therefore to name nothing. The documentation also records the blunt alternative and its cost: “Disabling EPT for virtual machines provides full mitigation for L1TF even with SMT enabled, because the effective page tables for guests are managed and sanitized by the hypervisor. Though disabling EPT has a significant performance impact especially when the Meltdown mitigation KPTI is enabled.” That is the whole trade-off in one sentence: shadow paging is safer here precisely because the hypervisor, not the guest, writes the entries the hardware walks.

AMD SEV-SNP attaches a different kind of policy, and its relationship to nested paging is worth stating carefully because it is often described as “a second nested page table” — it is not. The Reverse Map Table (RMP) is an inverse index:

“The Reverse Map Table (RMP) is a structure shared globally by all logical processors that resides in system memory and is used to ensure a one-to-one mapping between system physical addresses and guest physical addresses. Each page of physical memory that is potentially assignable to guests has one entry within the RMP. RMP entries contain the security attributes of the system physical page.” (APM Vol. 2 §15.36.3)

The nested page table is written by the hypervisor and therefore untrusted under SEV-SNP’s threat model; the RMP records, for each system physical page, which ASID and which guest physical address legitimately own it. The hardware checks both, and the ordering is the key architectural fact:

“All RMP checks described in this section occur after page table and nested page table access checks and have lower priority than existing paging checks.” (APM Vol. 2 §15.36.10)

So the NPT still decides whether the access is permitted and where it lands; the RMP then decides whether that landing is legitimate. A failed RMP check surfaces as a fault with a dedicated flag — “Bit 31 (RMP): Set to 1 if the fault was caused due to an RMP check or a VMPL check failure” on a #PF, and on #VMEXIT(NPF) additional EXITINFO1 bits: 34 (ENC) for the effective C-bit, 35 (SIZEM) for a PVALIDATE/RMPADJUST size mismatch, 36 (VMPL) for a VM privilege-level failure. The checks themselves — RMP-Covered, Hypervisor-Owned, Guest-Owned, Reverse-Map, Validated, Mutable, Page-Size, VMPL — include one that ties the two structures together directly: “Page-Size: … If the nested page table indicates a 2MB or 1GB page size, the Page_Size field of the RMP entry of the target page is 1.” The hypervisor cannot silently splinter or coalesce a confidential guest’s pages behind its back. KVM’s SEV-SNP support is present and default-on at v6.12: static bool sev_snp_enabled = true; module_param_named(sev_snp, sev_snp_enabled, bool, 0444); (arch/x86/kvm/svm/sev.c, v6.12).

Intel TDX takes the opposite structural approach: rather than an inverse index checked alongside a hypervisor-owned EPT, it introduces a Secure EPT owned by the TDX module rather than by the host kernel, which the host can only modify through SEAMCALL requests. At v6.12 this is not yet in KVM: arch/x86/kvm/vmx/tdx.c returns HTTP 404 at v6.12 through v6.15 and 200 at v6.16, so KVM’s ability to run TDX guests arrived in Linux 6.16. The mechanism it needed in the MMU is visible in the same file: tdp_mmu.c contains zero occurrences of “mirror” at v6.12 and twenty at v6.16, including void kvm_tdp_mmu_alloc_root(struct kvm_vcpu *vcpu, bool mirror) and a mirror_root_hpa field — the TDP MMU keeps a mirror of the Secure EPT that it can walk and reason about locally, and replays every structural change into the real table via the TDX module. (The host-side TDX module plumbing, arch/x86/virt/vmx/tdx/tdx.c, does exist at v6.12; it is the KVM guest support that is absent.)

Uncertain

Verify: the description of TDX Secure EPT semantics — specifically that the TDX module, not the host kernel, owns the Secure EPT and that all host modifications go through SEAMCALL. Reason: this claim is not sourced from the TDX module specification, which was not retrieved for this note. What is directly verified is the Linux-side evidence: the tag-by-tag existence check dating arch/x86/kvm/vmx/tdx.c to v6.16, and the appearance of mirror roots in tdp_mmu.c between v6.12 and v6.16. The Intel SDM chapter on SEAM (Ch. 35) was present in the manual read for this note but was not extracted. To resolve: read Intel SDM Vol. 3D Ch. 35 and the Intel TDX Module Architecture Specification, and confirm against Documentation/virt/kvm/x86/ at v6.16 or later. uncertain

Measured Overhead, Not Adjectives

Two-dimensional paging is usually described as “faster than shadow paging”, which is true on average and false often enough to matter. The primary measurements come from two sources: Bhargava et al.’s simulation of an AMD Opteron model (2008), and VMware’s two ESX evaluation papers (2009).

Bhargava’s Table 1 is the cleanest statement of the walk cost, isolated from everything else:

Benchmark suite	TLB misses per 100K instructions	2D walk latency ÷ native walk latency	Perfect-TLB headroom, native	Perfect-TLB headroom, 2D
MiscServer	294.3	4.01×	14.0%	75.7%
WebServer	129.0	3.90×	4.7%	44.4%
JavaServer	257.0	3.91×	13.5%	89.0%
IntCpu	70.4	4.57×	11.4%	48.6%
FpCpu	18.2	4.43×	5.7%	27.5%

Two-dimensional walk cost, from Bhargava et al., ASPLOS 2008, Table 1. The last two columns are the performance a perfect TLB would recover — i.e. an upper bound on how much of guest performance the walk is eating. What it shows: the paper’s own summary is “The slowdowns are significant, with nested walks being on average 3.9-4.6 times slower than their native counterparts”, and the headroom column says that on JavaServer, 89% of guest performance is theoretically recoverable from TLB and walk behaviour. The insight to take: 24 accesses does not mean 6× the walk latency — caching absorbs some of it — but roughly 4× is the measured reality, and on TLB-hostile server workloads that dominates everything else.

VMware’s measurements are of the end-to-end comparison a person actually cares about, and they carry three caveats that are routinely misquoted. The headline claims: EPT “provides performance gains of up to 48% for MMU-intensive benchmarks and up to 600% for MMU-intensive microbenchmarks” (Performance Evaluation of Intel EPT Hardware Assist, rev 20090330), and RVI “provides performance gains of up to 42% … and up to 500% for MMU-intensive microbenchmarks” (Performance Evaluation of AMD RVI Hardware Assist, rev 20090311).

The caveats: (1) every percentage is nested paging versus shadow paging, never versus bare metal — neither paper contains a virtualisation-versus-native overhead figure. (2) “48% better” is a time reduction, not a speedup: Apache Compile on four vCPUs goes from a normalised 0.48 to 0.25, so 0.25/0.48 = 0.52, a 48% shorter run and therefore a 1.92× speedup. (3) 48% versus 42% does not mean Intel beat AMD; VMware says so itself, because the two studies used different ESX versions (3.5 U2 versus a 4.0-era build) and different baselines (binary translation for the AMD paper, because ESX 3.5 supported AMD-V only with RVI).

The most useful number in either paper is not the headline. It is SPECjbb2005 with small pages, where nested paging is measured as a net loss:

Workload (1 vCPU, normalised; higher is better for jbb)	Shadow paging	EPT	Shadow, large pages	EPT, large pages
SPECjbb2005, 64-bit (Intel paper, Fig. 9)	1.00	0.85	1.14	1.14
SPECjbb2005, 32-bit (Intel paper, Fig. 8)	1.00	0.87	1.08	1.10
SPECjbb2005 (AMD paper, RVI vs BT)	1.00	0.88 (RVI)	1.22	1.25

The case where two-dimensional paging loses. What it shows: on a workload with a large working set and few page-table updates, EPT is 13–15% slower than shadow paging, and RVI 12% slower — and large pages erase the gap entirely. The insight to take: both papers draw the same conclusion in nearly the same words — “Because EPT further increases the TLB miss latency (due to additional paging levels), large page usage in the guest operating system is imperative for high performance of such applications in an EPT-enabled virtual machine.” This is the empirical counterpart to Bhargava’s 3.9–4.6× walk latency: shadow paging’s cheap 4-access walk still wins where walks dominate and page-table edits are rare, which is exactly the gap the Agile Paging work sets out to close.

The wins, for balance, are large and consistent where MMU maintenance dominates: Apache Compile at four vCPUs improves 48% (EPT) and 42% (RVI); Citrix XenApp improves about 30% and 29%; SQL Server Database Hammer about 12–13%; and the kernel microbenchmarks — fw (fork/exec) in particular, at 0.14 and 0.17 normalised — improve by roughly 6–7×, which is where the “600%” and “500%” headlines come from. Workloads with little MMU activity show nothing at all: Oracle Swingbench measured 1.00/1.00, 1.75/1.74, 2.70/2.71 across vCPU counts — identical to within noise.

Uncertain

Verify: whether these 2008–2009 figures still characterise current silicon. Reason: the measurements are from Nehalem-era Xeons and Shanghai-era Opterons on ESX 3.5/4.0, and both the hardware (much larger TLBs, deeper page-walk caches, page-walk-cache improvements across many generations) and the software (transparent huge pages by default, 1 GiB EPT mappings, the TDP MMU) have changed substantially. The shape of the result — walks are several times more expensive, huge pages close the gap, maintenance-heavy workloads win big — is corroborated by the mechanism and is unlikely to have inverted, but the specific percentages should not be quoted as current. To resolve: run perf kvm stat and perf stat -e dtlb_load_misses.walk_active on a modern host with and without kvm-intel.ept=0, or find a post-2020 primary measurement study. uncertain

Common Misunderstandings

“EPT/NPT replaces the guest’s page tables.” No — it adds a second table. The guest’s own page tables are untouched and fully functional; EPT/NPT only translates the GPAs they produce. A guest with paging disabled still goes through EPT.

“TDP is always faster than shadow paging.” Not for the walk itself — a TDP miss can cost ~24 accesses versus shadow’s 4. TDP wins because it eliminates the VM-exit storms that shadow paging suffered on page-table updates and context switches. Workloads with enormous TLB pressure but rare page-table changes are precisely where shadow paging’s cheap walk would still win, which is the gap the Agile Paging paper exploits.

“A guest-physical address is a real address.” It is a fiction the hypervisor manufactures; the corresponding host page can move, be swapped, or be shared at any moment.

“VPID/ASID and EPT/NPT are the same thing.” They are orthogonal: VPID/ASID tag TLB entries (the hit path); EPT/NPT define the walk (the miss path). A CPU can have one without the other.

“The Intel and AMD manuals say a nested walk costs 24 accesses.” Neither does. Intel says “requires multiple translations of guest-physical addresses using EPT”; AMD says “causes several nested table walks”. The number is from Bhargava et al. (ASPLOS 2008) as the general formula nm + n + m, restated by Gandhi et al. (ISCA 2016). Attributing it to a vendor manual is a citation error that this note previously made.

“EPT and NPT are the same feature with different names.” They solve the same problem with materially different architecture. Intel invented a new page-table entry format with a permission triple and no present bit — which is why “EPT misconfiguration” exists as a fault distinct from “EPT violation”. AMD reused the ordinary x86-64 page-table entry, which is why there is only one nested fault (#VMEXIT(NPF)) carrying a #PF-like error code. They also diverge on memory typing: Intel puts the type in the EPT leaf and switches MTRRs off for guest-physical accesses; AMD combines guest and host PAT types through a lattice and keeps MTRRs live, inventing the WC+ type to patch the coherency hole that creates.

“KVM’s TDP MMU replaced the shadow MMU.” It did not. At v6.12 both exist, and the KVM documentation states that a single VM “may contain pages from both TDP MMU and shadow MMU”. The shadow MMU still handles true shadow paging (no hardware TDP) and the nested case, where three logical translations must be squeezed into two hardware ones.

“An EPT misconfiguration means something is broken.” Usually the opposite: it means KVM is working. KVM deliberately writes architecturally-illegal 110b (write+execute, no read) entries to mark MMIO regions, precisely so that a guest access to a device address produces a distinct, high-priority exit. A high ept_misconfig count on a VM with lots of virtio traffic is expected, not pathological.

“Huge pages in the guest are enough.” Both dimensions splinter to the smaller size. AMD: “When an address is mapped by guest and nested page table entries with different page sizes, the TLB entry that is created matches the size of the smaller page.” A guest using transparent huge pages on 4 KiB-backed host memory gets 4 KiB TLB entries and, in Bhargava’s phrasing, “a splintered 2MB page in the guest could require as many as 512 4KB TLB entries.”

Production Notes

The performance signature of TDP is DTLB/STLB miss-heavy, page-walk-heavy workloads — large in-memory databases, big heaps, anything with a working set far larger than TLB reach. The standard mitigations, in order of impact: back guest RAM with huge pages (1 GiB pages give the largest TLB reach and shallowest walks); pin the guest’s memory to a single NUMA node to avoid cross-node walk latency; and ensure VPID/EPT (or ASID/NPT) are enabled (they are by default — confirm with cat /sys/module/kvm_intel/parameters/{enable_vpid,...} or the AMD equivalents). On Intel, perf kvm stat and the ept_violation / ept_misconfig exit counters reveal whether a VM is thrashing its EPT. For nested guests, the doubled translation (ngva→ngpa→gpa→hpa) makes huge pages and large memory pinning even more important.

A concrete tuning checklist, ordered by measured impact and each tied to a mechanism established above:

Lever	Mechanism it attacks	How to check it on a Linux host
Back guest RAM with 1 GiB hugetlbfs pages	Shrinks m in nm + n + m; 1 GiB on both sides takes 24 → 8	grep Huge /proc/meminfo; QEMU -mem-path /dev/hugepages
Enable THP in the guest too	Shrinks n; without it the host’s huge pages splinter to the guest’s 4 KiB	cat /sys/kernel/mm/transparent_hugepage/enabled inside the guest
Confirm hardware TDP is on	Without it KVM falls back to true shadow paging	cat /sys/module/kvm_intel/parameters/ept or /sys/module/kvm_amd/parameters/npt — both default Y
Confirm TLB tagging is on	Without VPID/ASID every transition flushes	cat /sys/module/kvm_intel/parameters/vpid
Confirm the TDP MMU is on	Read-mode mmu_lock instead of one write-held lock, for many-vCPU guests	cat /sys/module/kvm/parameters/tdp_mmu — default Y since 5.15
Confirm A/D bits and PML	Cheap dirty tracking during live migration instead of write-protect-and-fault	cat /sys/module/kvm_intel/parameters/{eptad,pml} — both default Y
Pin memory to one NUMA node	Cross-node latency multiplies through 24 accesses, not 4	numactl --membind, or libvirt <numatune>
Watch the exit counters	Distinguishes demand-faulting from MMIO thrash	perf kvm stat live, or the kvm:kvm_exit tracepoint filtered on reasons 48 (EPT_VIOLATION), 49 (EPT_MISCONFIG) and 62 (PML_FULL) — the numeric values are in uapi/asm/vmx.h, v6.12

Reading those exit counters is where diagnosis actually happens. A steady stream of EPT_VIOLATION early in a VM’s life is normal — it is guest RAM being demand-faulted in, one page at a time, exactly as designed. A sustained stream long after boot means something is repeatedly tearing SPTEs down: host memory pressure driving reclaim through the mmu_notifier path, KSM merging and un-merging, NUMA balancing migrating pages, or dirty logging active because a live migration is in progress. A high EPT_MISCONFIG rate is device traffic, not a fault — each one is an MMIO access to an emulated device, so it points at an I/O path that should probably be using virtio with a vhost backend. And PML_FULL appears only while dirty logging is on; if you see it outside a migration, something is holding the dirty-log mode open.

One last operational note on the security side. kvm-intel.ept=0 is a real mitigation for L1TF and is documented as such — “Disabling EPT for virtual machines provides full mitigation for L1TF even with SMT enabled” — but it forces every guest onto shadow paging, with the trap-on-every-page-table-write cost model that hardware TDP was built to eliminate, and the kernel documentation flags that the impact is “significant … especially when the Meltdown mitigation KPTI is enabled”. The default posture is the opposite: keep EPT, keep the SPTE address inversion, and use l1tf= and SMT control to manage the residual risk.

See Also

• Shadow Page Tables — the software predecessor TDP replaced; cheap walk, expensive maintenance (the inverse trade-off)
• Huge Pages and NUMA for Guests — the primary lever for cutting TDP walk cost
• Guest Physical Memory and Memory Slots — how KVM defines which GPA ranges map to which host memory
• Dirty Page Tracking and Live Migration — uses EPT write-tracking (write-protect → violation → log) for migration
• KSM and Memory Overcommit for VMs — dedup of guest pages underneath the EPT layer
• VM Exit Reasons and Handling — EPT violation/misconfig are two of the exit reasons
• Linux Memory Management MOC — the host MM subsystem that owns the HPAs EPT/NPT point at
• Nested Virtualization — the third translation (ngva→ngpa→gpa→hpa) that forces KVM back onto shadowing, this time of L1’s EPT
• Anonymous vs File-Backed Memory — what the HPAs on the far side of the EPT actually are, from the host’s point of view
• Linux Virtualization MOC — parent map (§4 Memory Virtualization)