Translation Lookaside Buffer
A Translation Lookaside Buffer (TLB) is the small, fast hardware cache that holds recent virtual-to-physical address translations so that the CPU does not have to walk the page table on every memory access. Every load, store, and instruction fetch first probes the TLB; a hit returns the translation in a single cycle, a miss triggers a page-table walk and refills the TLB with the result. The textbook framing is direct: a TLB is “a memory cache that stores the recent translations of virtual memory addresses to physical memory addresses” (per Wikipedia on TLB), making it part of the MMU. Without a TLB, every memory access would cost two to five extra memory accesses (one per page-table level) to resolve. With a typical 99%+ hit rate, the TLB hides almost all of that cost. This note explains the hit and miss flow, software-managed versus hardware-managed miss handling (RISC-V uses hardware), the structural choices (fully-associative, set-associative, multi-level), how ASID tagging avoids flushes on context switch, the
sfence.vmainstruction in RISC-V, TLB shootdown across multiple cores, the separation of I-TLB and D-TLB, the L1/L2 TLB hierarchy in modern CPUs, and typical sizes from real RISC-V and x86 cores.
The TLB lives in the memory protection and virtual memory section. For definitely-not-esp32 v1.0 there is no MMU and therefore no TLB; if Phase 7 adds an MMU running Sv32, the smallest workable TLB is a single fully-associative bank of 8 to 16 entries. This note is the conceptual reference for that future work and the explanation for why TLB design is one of the most studied parts of a CPU.
Mental Model
A page-table walk is a sequence of memory loads through nested tables. For Sv32, that is two loads per walk (one per level). For RISC-V’s 64-bit modes, three to five. The walk is correct but slow; on a hit in main memory, hundreds of cycles. The TLB is the cache that hides this cost by remembering the most recent walk results, keyed by virtual page number, returning the physical page number directly.
flowchart TD ACC["CPU emits virtual address<br/>(load/store/fetch)"] --> SPLIT{"split: VPN | offset"} SPLIT -->|"VPN"| TLB{"TLB lookup<br/>by VPN + ASID"} TLB -->|"hit"| HIT["combine PFN + offset<br/>= physical address<br/>(1-cycle)"] TLB -->|"miss"| WALK["page-table walk<br/>(hardware walker on RISC-V,<br/>~3 memory accesses for Sv39)"] WALK -->|"PTE found"| REFILL["evict an entry,<br/>install (VPN, ASID, PFN, perms)"] REFILL --> HIT WALK -->|"V=0, perm fail, etc."| FAULT["page-fault trap<br/>to S-mode handler"] HIT --> ACCESS["memory access<br/>or cache lookup"]
The TLB hit/miss flow. What it shows: every virtual address is split into VPN and offset; the VPN (plus the current ASID) is the key into the TLB, which either returns the physical frame directly or triggers a page-table walk; an unmapped entry or permission failure becomes a page-fault trap. The insight to take: the TLB is on the critical path of every memory access in the entire system, so its lookup latency must fit inside the L1-cache access window, which is why TLBs are tiny (tens of entries at L1) and fast.
The TLB is at the same architectural level as the L1 cache; conceptually it is the L1 cache for translations, sitting upstream of the L1 cache for data. The two work together: the VPN goes to the TLB, the resulting PFN combines with the offset, and the physical address goes to the L1 D-cache or I-cache. Modern designs overlap the TLB lookup and the cache tag check (a “virtually indexed, physically tagged” L1 cache); the offset bits index the cache while the TLB resolves the tag in parallel.
Hit and Miss Flow in Detail
On a hit. The TLB returns the physical page number (PFN) in one cycle. The CPU concatenates PFN with the page offset, gets the physical address, and proceeds to L1 cache lookup. Permission bits stored in the TLB entry are checked simultaneously; a permission violation triggers a page-fault even on a TLB hit.
On a miss. The walker takes over. On RISC-V, this is a hardware FSM that uses satp as its starting point and walks the page-table tree (see the Sv32 walk for the canonical algorithm). When a valid leaf PTE is found, the walker installs an entry in the TLB (evicting an existing one if needed) and resumes the original access. When the walk fails (invalid PTE, permission violation, misaligned superpage), the walker raises a page-fault exception and the OS handles it.
OSTEP frames the cost dramatically: a TLB miss can incur “a factor of ~100x slowdown” compared to a hit (per OSTEP TLB chapter). The exact number depends on where the page table lives in the cache hierarchy. An L1-cache-resident walk costs three cycles per level; a DRAM-resident walk costs hundreds.
The reason TLB hit rates are typically above 99% is locality. Programs access memory in clusters; a 4 KiB page covers a thousand pointers. As OSTEP puts it (per vm-tlbs.pdf), “Programs exhibit both temporal and spatial locality in their memory access patterns. This characteristic explains why TLB hit rates are remarkably high.” A loop touching successive elements of an array refills the TLB once per page boundary; everything else hits.
Software-Managed Versus Hardware-Managed TLBs
When the TLB misses, who walks the page table?
Hardware-managed TLB. The CPU has a built-in page-table walker that knows the table format. On a miss, the walker fires, fetches PTEs from memory, installs the result, and resumes. The OS never sees the miss; it just sees an occasional pause when the walker stalls on a long memory chain. This is the design used by x86, ARM, and RISC-V (per supervisor.adoc, Sv32 design rationale: “we have architected page table layouts to support a hardware page-table walker”). The cost is hardware area and a fixed table format the OS must match.
Software-managed TLB. The CPU has no walker. On a TLB miss, the CPU raises a special TLB-miss trap to a fast OS handler, which walks any data structure it likes and writes the resulting translation into the TLB via privileged instructions. The OS is free to use any table format: hash tables, B-trees, custom layouts. The cost is a real trap on every miss, which is slower than a hardware walk. MIPS, classic SPARC, and Alpha used this model.
The RISC-V spec adds a useful nuance: “An implementation can choose to implement software TLB refills using a machine-mode trap handler as an extension to M-mode” (per supervisor.adoc). The base architecture mandates a hardware walker; an implementor could trap on miss in M-mode and software-walk, but no shipping cores do this. The OS-visible behavior is hardware-walked.
Modern thinking has settled on hardware-managed for two reasons. First, TLB miss handling competes with the OS’s own work; trapping into the kernel hundreds of thousands of times a second is costly. Second, hardware walkers can speculatively prefetch translations and issue PTE fetches in parallel, which software handlers cannot. The OSTEP framing was once “RISC = software-managed, CISC = hardware-managed” but RISC-V breaks the pattern; the rule of thumb is now “hardware-managed unless you have a very specific reason.”
TLB Entry Contents
A TLB entry is essentially a cached PTE with some extra bookkeeping. A typical entry contains:
- VPN: the virtual page number (key for lookup).
- ASID: the address-space identifier (so entries from different processes coexist).
- PFN / PPN: the physical page number (the translation result).
- Permission bits: R, W, X, U, plus possibly D, A, G, copied from the source PTE.
- Valid bit: distinguishes an empty slot from a real translation.
- Global bit (G): if set, the entry matches regardless of ASID, supporting kernel mappings that are shared across all processes.
Some designs also cache the page size (4 KiB, 2 MiB, 1 GiB superpage) so the entry covers multiple base pages with one TLB slot.
Fully-Associative Versus Set-Associative
Small L1 TLBs are typically fully associative: every entry has its own comparator, every lookup compares against every entry. This is the textbook design Wikipedia describes (per Wikipedia on TLB) and matches the SiFive U54’s “32-entry fully associative” ITLB and DTLB (per SiFive U54-MC manual). Fully-associative is the most flexible (any VPN can occupy any slot) but does not scale; 1024 comparators is too many to fit in the critical path.
Large L2 TLBs are set-associative. The VPN is hashed (typically by simple bit-extraction) into one of S sets, each containing K ways; the lookup compares against the K entries in the chosen set. Intel Skylake’s L2 STLB has 1536 entries and is “12-way set associative” (per WikiChip Skylake). A set-associative TLB trades some flexibility (a VPN can only land in K slots) for area and timing.
The same trade-off as data caches: fully-associative for the small fast one, set-associative for the larger slower one.
ASIDs and Avoiding Flush on Context Switch
A naive TLB has a problem on context switch. Two processes can have the same virtual address mapped to different physical pages; if the TLB retains translations from process A, process B’s first accesses will see A’s mappings. The textbook fix is to flush the entire TLB on every context switch. The performance cost is significant: every subsequent access to a heavily-used page costs a fresh walk.
The ASID tag solves this. Each TLB entry carries the ASID of the process whose page table produced it. The lookup compares VPN and ASID; entries with the wrong ASID are ignored. On context switch, the kernel writes the new process’s ASID into the ASID field of satp; the old entries remain but are invisible until the kernel switches back. No flush required.
RISC-V’s ASID field in satp is up to 9 bits on Sv32 (ASIDMAX = 9) and up to 16 bits on Sv39/48/57 (ASIDMAX = 16) (per supervisor.adoc, “Supervisor Address Translation and Protection (satp) Register”). The implementation may implement fewer bits than ASIDMAX, and may implement zero (in which case the kernel must flush the TLB on every context switch). Software determines ASIDLEN by writing all-ones into the ASID field and reading back what stuck.
x86 calls the same feature PCID (Wikipedia on TLB: “Intel 64 processors since Westmere support 12-bit PCIDs”); Alpha called them ASNs; ARM has it under the name ASID as well. The discipline is the same in each: number address spaces, tag TLB entries, switch by writing the number.
ASID recycling is the corollary problem. The ID space is finite. When the kernel runs out of free ASIDs, it must reuse one. Before reusing, the kernel must execute sfence.vma x0, asid to flush all stale entries for the recycled ASID, otherwise the next access by the new owner would see the previous owner’s translations (per supervisor.adoc: “When software recycles an ASID … it should execute SFENCE.VMA with rs1=x0 and rs2 set to the recycled ASID”).
sfence.vma in RISC-V
sfence.vma is RISC-V’s TLB management instruction. It serves two purposes (per supervisor.adoc, “Supervisor Memory-Management Fence Instruction”): it orders previous stores to memory-management data structures before subsequent implicit references to them, and it invalidates address-translation cache (TLB) entries.
The instruction takes two register operands, with x0 as a wildcard:
sfence.vma x0, x0: invalidate all entries for all ASIDs. The big hammer.sfence.vma x0, asid: invalidate all entries for the given ASID, except global mappings.sfence.vma va, x0: invalidate leaf-PTE entries for the given virtual address, for all ASIDs.sfence.vma va, asid: invalidate leaf-PTE entries for the given va in the given ASID, except global mappings.
The spec is deliberate about calling this a fence and not a TLB flush, with this rationale (per supervisor.adoc): “The SFENCE.VMA is used to flush any local hardware caches related to address translation. It is specified as a fence rather than a TLB flush to provide cleaner semantics with respect to which instructions are affected by the flush operation and to support a wider variety of dynamic caching structures and memory-management schemes.”
The wording matters because RISC-V is weakly ordered: stores to page tables become visible to the page-table walker only after a fence forces the cache hierarchy to commit. The RISC-V community thread on the design (groups.riscv.org isa-dev SFENCE.VM thread) makes the point directly: a memory fence alone “may not be strong enough to propagate PTE changes to the page table walk mechanism.” sfence.vma is the explicit synchronization.
Crucially, sfence.vma is local to the issuing hart. It does not propagate to other harts.
TLB Shootdown on Multi-Core
A multi-core system has one TLB per hart. When the kernel modifies a PTE, every hart that may have cached the old translation must be told to invalidate. Since RISC-V’s sfence.vma is local, the kernel must use an inter-processor interrupt (IPI) to reach remote harts.
The classic shootdown protocol (per the spec commentary and Linux’s tlbflush.c):
- The initiating hart writes the PTE update to memory.
- The initiating hart executes a local
sfence.vmato flush its own caches and to globally publish the writes. - The initiating hart sends an IPI to every other hart that may have cached the entry. (The kernel tracks “which harts have this
mmactive” via a per-mm CPU mask.) - Each remote hart, upon receiving the IPI, executes
sfence.vmain its handler to flush the relevant entries. - Each remote hart acknowledges. The initiating hart waits for all acknowledgments before proceeding.
The Linux kernel implements two paths in arch/riscv/mm/tlbflush.c:
- SBI-mediated:
sbi_remote_sfence_vma_asidasks the M-mode firmware to dispatch the fence on the listed remote harts. Useful in M-mode-controlled setups where IPIs are routed through SBI. - Direct IPI:
on_each_cpu_mask()sends an IPI to each target hart; the handler executes a localsfence.vma.
The code uses riscv_use_sbi_for_rfence() to pick at boot.
TLB shootdown is expensive. The general analysis (linuxvox blog on TLB shootdown) calls out IPI cost: “sending an IPI to 100 cores takes microseconds (or longer).” At hundreds of cores, shootdown becomes a serious bottleneck. ARM’s tlbi vae1is does the broadcast in hardware; x86 needs IPIs like RISC-V; both pay the cost. RISC-V is working on an Svinval extension that adds finer-grained, batch-friendly variants of sfence.vma (the local_sfence_inval_ir() and local_sinval_vma() helpers in tlbflush.c are precisely those Svinval variants) and a Sstc / hardware-broadcast extension is in discussion.
Split I-TLB / D-TLB and Multi-Level Hierarchy
Modern CPUs split the L1 TLB into a separate I-TLB (for instruction fetches) and D-TLB (for loads and stores). The rationale is the same as for the split L1 I-cache and D-cache: instruction and data access patterns are different, and two specialized smaller TLBs are faster than one bigger one.
A unified L2 TLB sits behind both, catching misses from either. The L1 TLBs are fully associative, the L2 is set-associative, and a miss in L2 falls through to the hardware walker.
Concrete sizes from shipping hardware:
| Core | I-TLB (entries) | D-TLB (entries) | L2 TLB (entries) | Walker |
|---|---|---|---|---|
| SiFive U54 (manual) | 32 fully-assoc | 32 fully-assoc | none | hardware |
| Intel Skylake (WikiChip) | 128 (4-way) | 64 (4-way, 4 KiB) | 1536 (12-way) | hardware |
| Intel Nehalem (per Wikipedia) | 128 entries | 64 entries (4 KiB) | 512-entry unified L2 | hardware |
The U54’s 32-entry single-level TLB is at the low end; it suffices for the embedded workloads the U54 targets. Skylake’s 1536-entry L2 is at the high end; it is sized to cover the working set of a server-class workload.
Where the L1 TLBs have separate banks for different page sizes (Skylake has 64 entries for 4 KiB pages, 32 for 2 MiB pages, 4 for 1 GiB pages, per 7-cpu Skylake notes), the OS chooses the page size by allocating an appropriately-aligned region; transparent huge pages in Linux do this automatically.
TLB Misses Versus Page Faults
These are distinct and easily confused.
A TLB miss is a hardware event: the translation was not in the TLB. It is resolved automatically by the page-table walker. The OS does not see it (on RISC-V); it manifests only as extra latency on the missing access.
A page fault is a software event: the page-table walk found that the page is not mapped, or has insufficient permissions for the access. It raises a trap to the OS. The OS handler does something (paging in from swap, COW, deny and signal), then returns; the instruction re-executes.
A TLB miss does not always lead to a page fault; most TLB misses are resolved by the walker and the access succeeds. A page fault always involves the walker (it is what reports the failure), but the walker’s failure is the cause, not the TLB miss itself.
Failure Modes
TLB poisoning. The kernel writes a new PTE; an existing TLB entry from before the write survives. The next access still uses the stale translation. The fix is sfence.vma after every PTE modification, with the right operands.
Forgotten shootdown. The kernel writes a new PTE and fences its own TLB but not remote harts’. On a multi-core system, the other harts continue using stale translations. The fix is to track which harts have the affected mm active and IPI them.
ASID aliasing. The kernel reuses an ASID without flushing. The new owner sees stale entries from the previous owner. The fix is sfence.vma x0, asid on recycle.
Global mapping mismarked. A kernel mapping (G=1) survives an ASID-tagged flush. If the mapping should have been per-process, the wrong page is now visible across address spaces. The spec is explicit: “marking a non-global mapping as global is a software bug that, after switching to an address space with a different non-global mapping for that address range, can unpredictably result in either mapping being used” (per supervisor.adoc).
Multiple TLB entries for the same VPN. Cached non-leaf and leaf entries can coexist; if a page is upgraded to a megapage without invalidating the non-leaf entry first, the TLB can match both, and which one is used is implementation-defined (per supervisor.adoc). The fix is sfence.vma between break-before-make sequences.
Speculative caching. RISC-V allows the walker to populate the TLB speculatively (per supervisor.adoc: “Implementations may also execute the address-translation algorithm speculatively at any time, for any virtual address, as long as satp is active”). Code that examines the state of the world before a fence and assumes “no fence yet, so no TLB entry yet” is wrong.
Production Notes
Linux RISC-V. The tlbflush.c file in arch/riscv/mm/ (source) implements the full ladder: flush_tlb_page, flush_tlb_range, flush_tlb_mm, each choosing between SBI rfence calls and direct IPIs based on platform support. The Svinval extension is detected via has_svinval() and used to issue more efficient sequence-able invalidations when available.
Spectre and friends. TLB and translation caches have been a target of side-channel attacks (Meltdown was partly a TLB-affecting flaw). Mitigations like KPTI (Kernel Page-Table Isolation) flush the TLB on user/kernel transitions, dramatically increasing TLB pressure. ASID / PCID support is the standard mitigation against the resulting cost.
Open RISC-V cores. CVA6’s TLB (CVA6 MMU documentation) is a small fully-associative bank with ASID tagging, parameterized at synthesis. VexRiscv and other open RV32 cores often skip the TLB entirely in their smallest configurations.
definitely-not-esp32 roadmap. v1.0 has no TLB (no MMU). Phase 7 stretch goal: a 16-entry fully-associative TLB sufficient for Sv32 with a hardware walker. Shootdown is moot in a single-core SoC; the local sfence.vma is enough.