vault

Root Mode and Non-Root Mode

12 min read

Root Mode and Non-Root Mode

When Intel VMX (VT-x) and AMD SVM (AMD-V) added hardware virtualization to x86, they did not add a new privilege ring. They added an entirely new, orthogonal dimension of privilege that sits underneath the existing ring 0–3 model. Intel calls the two halves of this dimension VMX root operation and VMX non-root operation; AMD calls them host mode and guest mode (Intel SDM Vol 3C §23; Wikipedia: x86 virtualization). The hypervisor runs in root/host operation; the guest runs in non-root/guest operation — and each side has its own complete, independent set of rings 0, 1, 2, 3. The defining insight, the one this whole note exists to drive home, is that the guest kernel really does run in its own ring 0 — fully privileged from the guest’s point of view — yet a “sensitive” operation it performs still traps out to the hypervisor, because the trap is triggered by crossing the root/non-root boundary, not by a ring-level privilege check. This is a second axis of privilege, not a relabelling of the first.

This corrects the single most common misconception about hardware virtualization: that the guest “runs in ring 1,” or that the hypervisor “is ring -1” in any literal sense. The “ring -1” phrase is a useful metaphor — the hypervisor is more privileged than the guest’s ring 0 — but it is not how the hardware is built. The hardware is built as a two-dimensional grid: (root vs non-root) × (ring 0..3). This note explains that grid, how a CPU moves between its two halves, and why guest ring 0 is simultaneously “fully privileged” and “still trappable.” The instruction-level details of how you enter/leave each mode live in Intel VMX and AMD SVM and VM Entry and VM Exit Mechanics; this note is about the privilege model itself.

Mental Model

flowchart TB
  subgraph ROOT["VMX root operation / SVM host mode (the hypervisor)"]
    R0["Ring 0: KVM, host kernel"]
    R3R["Ring 3: QEMU / VMM userspace"]
  end
  subgraph NONROOT["VMX non-root operation / SVM guest mode (the guest)"]
    G0["Ring 0: guest kernel (privileged WITHIN the guest)"]
    G3["Ring 3: guest user processes"]
  end
  R0 -->|"VM entry: VMLAUNCH/VMRESUME (VMX) or VMRUN (SVM)"| G0
  G0 -->|"VM exit / #VMEXIT: sensitive op traps OUT"| R0
  G3 -.->|"ordinary syscall stays inside guest"| G0

The two-dimensional privilege grid. What it shows: there are two independent ring stacks — one for the hypervisor (root/host) and one for the guest (non-root/guest). A guest user process making an ordinary system call goes from guest ring 3 to guest ring 0 and stays inside the guest (dashed arrow) — the hypervisor never sees it. But when guest ring 0 executes a sensitive operation, the CPU performs a VM exit that crosses the vertical boundary up into hypervisor ring 0 (solid up-arrow), regardless of the fact that the guest was already at its highest ring. The insight to take: privilege is now a pair (mode, ring). Guest ring 0 is the top of the guest’s own stack but is below the entire root/host plane; that vertical gap is what every VM exit jumps across.

Why a Second Axis Was Necessary

The classical way to virtualize a CPU is trap-and-emulate: run the guest deprivileged, let every sensitive instruction fault, and emulate it in the hypervisor (see Trap-and-Emulate and the Popek-Goldberg Requirements). On x86 this was impossible with rings alone because the guest kernel expects to run in ring 0, and a set of x86 instructions behaved differently in user mode without faulting — they failed silently rather than trapping (see Why x86 Needed Hardware Virtualization). Two pre-hardware workarounds existed, and understanding their pain explains why the second axis was the right fix.

The first was ring deprivileging (Xen’s original paravirtualization, and the “0/1/3 model”): move the guest kernel from ring 0 down into ring 1, keep the hypervisor in ring 0, and keep guest userspace in ring 3. This almost works, but it breaks in two ways. Sensitive-but-non-privileged instructions still do not trap from ring 1. And the guest kernel can observe that it is not in ring 0 (by reading its code-segment selector’s privilege level), shattering the illusion — so the guest had to be modified (paravirtualized) to tolerate it. The second workaround was binary translation (VMware), rewriting the guest’s kernel instruction stream on the fly to force traps — correct but slow and enormously complex.

VT-x/SVM cut the knot by adding the second axis. Now the hypervisor does not have to steal ring 0 from the guest. The guest kernel keeps its rightful ring 0 — in non-root/guest mode — and runs unmodified. The hypervisor lives in a different mode (root/host), so there is no ring conflict at all. The trapping problem is solved not by deprivileging but by configuring, per-VM, exactly which sensitive operations cause a VM exit across the mode boundary. The Intel SDM’s own framing captures it: a “new VMX non-root class, functionally containing a standard x86 ring-0, 1, 2, 3 privilege model, has been added… the guest operating systems run in their intended privileged execution mode” (summarized from Intel SDM Vol 3C).

How the CPU Moves Between the Two Modes

A logical processor is in one of three states with respect to virtualization: not in VMX/SVM operation at all (a freshly booted CPU), in root/host operation (the hypervisor armed the CPU but no guest is currently running on it), or in non-root/guest operation (a guest is executing). The transitions are:

  • Arming the CPU moves it from “no VMX/SVM” into root/host operation. On Intel this is VMXON after setting CR4.VMXE; on AMD it is setting EFER.SVME. (Full instruction detail in Intel VMX and AMD SVM.) An Intel community engineer’s distinction is worth internalizing: a CPU that has executed VMXON is in VMX root operation even when no guest is running — root operation is “the hypervisor is in charge” state, distinct from the pre-VMXON state where VMX is simply off (Intel Community).
  • VM entry moves the CPU from root/host into non-root/guest. Intel: VMLAUNCH/VMRESUME. AMD: VMRUN. The CPU loads the guest’s full register and control state (including which ring the guest resumes in) from the control structure (VMCS and VMCB (Virtual Machine Control Structure)) and begins running guest instructions directly on the silicon. The SDM names these “VM entries” and the reverse “VM exits,” collectively “VMX transitions.”
  • VM exit (Intel) / VMEXIT (AMD) moves the CPU from non-root/guest back to root/host. This happens when the guest does something the hypervisor configured as intercepted: executes a sensitive instruction, touches a device register, takes an interrupt, or voluntarily calls out (VMCALL/VMMCALL). The hardware writes an exit reason/EXITCODE and resumes the hypervisor. Crucially, a VM exit is not a ring transition within the guest — it is a mode transition. The guest may have been in its ring 0 or its ring 3; either way, the exit lands the CPU in the hypervisor’s root-mode ring 0.

The key consequence: most of what a guest does never causes a VM exit at all. A guest user program calling read() traps from guest ring 3 to guest ring 0 using the guest’s own IDT and syscall MSRs, runs the guest kernel’s handler, and returns — entirely within non-root/guest mode, at full native speed, with the hypervisor completely uninvolved. Only operations that pierce the mode boundary cost an exit. The entire performance engineering of KVM — virtio, vhost, APICv, posted interrupts, two-dimensional paging — is, at bottom, the art of configuring fewer operations to cross this boundary. See the cross-cutting theme in Linux Virtualization MOC: “it’s all about eliminating the VM exit.”

Worked Example: Two Kinds of Privileged Operation in the Guest

Consider a guest Linux kernel (running in guest ring 0) performing two different privileged operations, to make the two-axis model concrete.

  1. mov %rax, %cr3 (reload the page-table base — a routine context switch inside the guest). The guest kernel is in its ring 0, so the ring check passes; it does not get a #GP for insufficient privilege. Whether this causes a VM exit depends on hypervisor configuration. With hardware Two-Dimensional Paging (EPT and NPT) enabled, the guest owns its own CR3 and this write does not exit — the guest manages its own page tables freely, and the second-level (EPT/NPT) translation handles guest-physical→host-physical transparently. Here, guest ring 0 is genuinely privileged and the hypervisor stays asleep.

  2. wrmsr to a model-specific register the hypervisor wants to police, or an access to an emulated device’s MMIO region. The ring check also passes (guest ring 0 is allowed to wrmsr), but the operation is configured as intercepted, so the CPU performs a VM exit. The hypervisor inspects the exit reason, emulates the effect (perhaps adjusting a virtual register, perhaps faulting the guest), and resumes it. Here, guest ring 0 was privileged within the guest yet still trapped out of the guest.

The same instruction-from-ring-0 yields opposite behaviors not because of any ring difference — both ran in guest ring 0 — but because of the mode-boundary policy the hypervisor programmed. This is the entire point of the second axis: privilege within the guest and trappability to the hypervisor are now independent knobs.

“Ring -1”: A Metaphor, Not a Hardware Mode

The phrase “ring -1” for the hypervisor is widespread and intuitively reasonable — the hypervisor is more privileged than the guest’s ring 0, so it sits “below” ring 0. But it is not an architectural ring. There is no CPL = -1; the hypervisor’s own code runs in ordinary ring 0 (KVM is kernel code) and ring 3 (the QEMU/VMM userspace process), exactly the rings everything else uses. What makes it “more privileged” is that it runs in root/host mode, the side of the orthogonal axis that controls the non-root/guest side. So “ring -1” is shorthand for “root mode is above the entire guest ring stack.” Using it carefully is fine; believing the CPU has a literal negative ring is the misconception. (For comparison, “System Management Mode” is sometimes called “ring -2” — also a metaphor for being even more privileged, not a real ring.)

Uncertain

Verify: the precise SDM phrasing that VMX non-root operation contains a “standard x86 ring-0,1,2,3 privilege model” and that guests “run in their intended privileged execution mode.” Reason: this exact wording was obtained via a secondary summary of the Intel SDM rather than transcribed from the SDM PDF directly (the PDF was located but the specific sentence not verbatim-extracted). To resolve: confirm against Intel SDM Vol 3C, Chapter 23 “Introduction to Virtual Machine Extensions,” §23.1–23.3 (“VMX Operation,” “VMX root vs non-root”). uncertain

Failure Modes and Common Misunderstandings

“The guest kernel runs in ring 1.” False for VMX/SVM, as established above. This was true of paravirtualized Xen on 32-bit x86 (ring deprivileging), and the idea persists by inertia. On hardware-assisted virtualization the guest kernel runs in its own ring 0 in non-root/guest mode.

“A VM exit is like a system call / a ring transition.” Misleading. A syscall is a ring transition (3→0) within one mode. A VM exit is a mode transition (non-root→root) that is largely independent of which guest ring was active. The two are different axes; conflating them obscures why a guest can do most ring-0 work without ever exiting.

“Root mode means a guest is running.” No. A CPU in VMX root operation may have no guest currently scheduled on it — root operation is the hypervisor’s normal armed state. The guest runs only in non-root operation, between a VM entry and the next VM exit. The Intel Community thread makes exactly this distinction between “VMX root mode” and “VMX OFF’d” (Intel Community).

Assuming root and non-root share register/control state. They do not, and that is the whole reason a control structure exists. The CPU swaps the entire architectural state (rings, segment registers, control registers, the active page-table root) across every mode transition by saving/loading the VMCS/VMCB. A bug where host and guest state bleed across the boundary is a security catastrophe — see the SVM VMSAVE/VMLOAD discussion in Intel VMX and AMD SVM and the real-world breakout in Google Project Zero’s “An EPYC escape.”

Intel vs AMD Terminology — Same Concept, Different Words

The two vendors describe the identical concept with different vocabulary, and reconciling them prevents confusion when reading mixed documentation:

ConceptIntel (VMX)AMD (SVM)
Hypervisor’s half of the axisVMX root operationhost mode
Guest’s half of the axisVMX non-root operationguest mode
Enter the guestVM entry (VMLAUNCH/VMRESUME)VMRUN (host→guest world switch)
Leave the guestVM exit#VMEXIT
Both transitions, collectivelyVMX transitionsworld switch

The substance is identical: a hypervisor half that controls a guest half, each with full rings 0–3, separated by a hardware boundary that the guest crosses on intercepted operations. AMD’s “world switch” wording for VMRUN/#VMEXIT nicely captures that the entire CPU context flips between two complete worlds.

See Also

Graph View

Backlinks