Memory Barriers in the Linux Kernel

A memory barrier is an instruction (or compiler directive) that constrains the order in which one CPU’s memory accesses become visible to other CPUs and to devices. Modern processors and compilers freely reorder, defer, combine, and speculate loads and stores for performance; a single CPU never notices because it always sees its own accesses in program order, but a second observer can perceive them in a scrambled order. As Documentation/memory-barriers.txt puts it, barriers “impose a perceived partial ordering over the memory operations on either side of the barrier” and are used “to override or suppress these tricks, allowing the code to sanely control the interaction of multiple CPUs and/or devices” (per the v6.12 doc). The kernel exposes a small, deliberate vocabulary of these primitives — full, read, and write barriers; acquire/release pairs; and device/DMA barriers — and pairs them in stereotyped patterns. Get the pairing wrong and a reader can observe a published pointer before the data it points to. This note is the API reference and catalogue — every barrier macro the kernel exposes, what each one is for, and exactly what each compiles to on x86-64, arm64, PowerPC, RISC-V and s390. The hardware reason barriers exist (cache coherence, the store buffer, the invalidate queue) lives in Cache Coherence and the Store Buffer; the semantics of what any one of them promises lives in Acquire Release and Fence Semantics; the formal model that makes those promises machine-checkable is The Linux Kernel Memory Model.

This note pins to Linux 6.12 LTS (released 2024-11-17; a maintained long-term-support branch — mainline had moved into the 7.x series by the time of writing, 2026-09-04), verifying every macro definition against the raw v6.12 source blobs listed in sources. Every arch lowering quoted below was read from the tree with curl, not from memory.

Scope: This Note Is the Catalogue

Memory ordering in this vault is split across four notes, and the split is deliberate because the subject genuinely has four separable layers. Naively deepening any one of them re-derives the others, so the boundary is stated here explicitly and the siblings state the mirror image of it in their own scope sections.

flowchart TB
  HW["<b>[[Cache Coherence and the Store Buffer]]</b><br/>THE MACHINE — why reordering exists<br/>store buffer, invalidate queue, MESI,<br/>x86-TSO vs weak ARM/POWER/RISC-V"]
  CT["<b>[[Acquire Release and Fence Semantics]]</b><br/>THE CONTRACT — what you are promised<br/>one-way barriers, reads-from pairing,<br/>litmus tests, A-cumulativity, RCpc vs RCsc"]
  FM["<b>[[The Linux Kernel Memory Model]]</b><br/>THE FORMALISATION — machine-checkable<br/>cat files, herd7, ppo / hb / prop"]
  CAT["<b>THIS NOTE</b> — THE CATALOGUE<br/>every macro, what it is FOR,<br/>and what it COMPILES TO"]
  L1["Mandatory mb / rmb / wmb<br/>(always emit an instruction)"]
  L2["dma_mb / dma_rmb / dma_wmb<br/>(CPU ↔ device, coherent memory)"]
  L3["virt_mb / virt_rmb / virt_wmb<br/>(UP guest ↔ SMP host)"]
  L4["pmem_wmb, io_stop_wc,<br/>smp_mb__before/after_atomic,<br/>smp_store_mb, smp_cond_load_acquire,<br/>barrier_nospec, array_index_nospec"]
  L5["The readX/writeX I/O accessors<br/>and their five ordering guarantees"]
  L6["The per-architecture lowering table"]
  HW -->|"motivates"| CT
  CT -->|"is made precise by"| FM
  CAT --> L1 & L2 & L3 & L4 & L5 & L6
  CT -.->|"defers the full<br/>macro inventory to"| CAT
  HW -.->|"defers the full<br/>macro inventory to"| CAT

The four-note division of memory ordering, and what this note owns. What it shows: three notes answer why, what is promised, and how it is formalised; this one answers which macro, and what does it cost here. The six boxes on the right are the material no sibling covers. The insight to take: the division is not arbitrary — the two dotted arrows are quoted from the siblings themselves. Acquire Release and Fence Semantics names this note as “the catalogue of every barrier macro including the dma_*() and mandatory (mb()/rmb()/wmb()) families this note does not cover”, and Cache Coherence and the Store Buffer names it as “the full barrier catalogue: mandatory versus SMP-conditional, dma_*(), pmem_wmb(), and the per-architecture lowering in detail”. This note is written to that contract.

Concretely: if your question is “which macro do I write, and what will it cost on arm64?”, you are in the right note. If it is “will smp_load_acquire() order this store against that load?”, read Acquire Release and Fence Semantics. If it is “why does my code work on my laptop and corrupt a list on an Ampere server?”, read Cache Coherence and the Store Buffer. If it is “how do I prove this pattern is safe?”, read The Linux Kernel Memory Model.

The consequence of that boundary for this note is that the semantics sections below are deliberately brief — enough to make the catalogue usable without opening a sibling, and no more. The weight is on the inventory, the dispatch machinery, the per-architecture lowering, and the device- and virtualisation-facing families that the semantics notes never touch because they are not about CPU-to-CPU ordering at all.

Mental Model: A Line in the Access Queue

The cleanest way to think about a barrier is the kernel doc’s own metaphor: it “draws a line in that CPU’s access queue that accesses of the appropriate type may not cross.” A CPU issues loads and stores into a queue of pending memory operations and is free to let later operations overtake earlier ones. A barrier inserts a fence into that queue: operations of the relevant type that appear before the barrier in program text are forced to take effect, as seen by other observers, before any operation of the relevant type that appears after it.

Crucially, a barrier on one CPU does nothing on its own. The doc is blunt: “There is no guarantee that issuing a memory barrier on one CPU will have any direct effect on another CPU.” A barrier only orders the accesses of the CPU that executes it. Ordering between two CPUs emerges only when both sides cooperate — the writer fences its stores and the reader fences its loads. This is the single most important idea in the whole subject, and it is why barriers come in pairs.

flowchart LR
  subgraph W["Writer CPU"]
    direction TB
    W1["STORE data = 42"] --> WB["smp_wmb()<br/>write barrier"]
    WB --> W2["STORE ready = 1"]
  end
  subgraph R["Reader CPU"]
    direction TB
    R1["LOAD ready"] --> RB["smp_rmb()<br/>read barrier"]
    RB --> R2["LOAD data"]
  end
  W2 -. "ready propagates" .-> R1
  RB -. "ordering only<br/>holds if BOTH<br/>barriers present" .-> WB

The canonical publish-then-flag / read-flag-then-data pattern. What it shows: the writer separates the data store from the flag store with smp_wmb(); the reader separates the flag load from the data load with smp_rmb(). The insight: the two barriers must be present together — they are a matched pair. The writer’s smp_wmb() guarantees data=42 is committed before ready=1; the reader’s smp_rmb() guarantees that once it has seen ready==1, the subsequent load of data cannot be satisfied from a stale value. Remove either barrier and the reader can legally observe ready==1 while data is still the old value.

The Complete Vocabulary

Before any semantics, here is the whole inventory in one place, because “which one of these thirty macros do I want?” is the question this note exists to answer. Every entry was read from include/asm-generic/barrier.h, include/asm-generic/rwonce.h, include/linux/compiler.h, include/linux/nospec.h, include/linux/rcupdate.h and include/linux/spinlock.h at v6.12.

The organising idea is that a barrier is defined by whose view of memory it constrains. There are five different observers a kernel programmer might need to order against, and they need different instructions:

flowchart TB
  Q["What am I ordering<br/>my accesses against?"]
  O1["<b>The compiler</b><br/>— it will reorder, fuse,<br/>invent and tear my accesses"]
  O2["<b>Another CPU</b><br/>in this coherency domain"]
  O3["<b>A DMA device</b><br/>reading coherent memory<br/>(not a coherency participant)"]
  O4["<b>An MMIO register</b><br/>on a device bus,<br/>outside the cache system"]
  O5["<b>A hypervisor host</b><br/>that may be SMP even<br/>though my guest is UP"]
  O6["<b>Persistence</b><br/>— has the store reached a<br/>platform durability domain?"]
  A1["barrier()<br/>READ_ONCE / WRITE_ONCE"]
  A2["smp_mb / smp_rmb / smp_wmb<br/>smp_store_release / smp_load_acquire<br/>smp_mb__before_atomic / __after_atomic"]
  A3["dma_mb / dma_rmb / dma_wmb"]
  A4["mb / rmb / wmb (mandatory)<br/>or the readX/writeX accessors"]
  A5["virt_mb / virt_rmb / virt_wmb"]
  A6["pmem_wmb()"]
  Q --> O1 & O2 & O3 & O4 & O5 & O6
  O1 --> A1
  O2 --> A2
  O3 --> A3
  O4 --> A4
  O5 --> A5
  O6 --> A6
  NOTE["Collapse rule: on a UP build the smp_* family<br/>degrades to barrier(); the mandatory, dma_* and<br/>virt_* families do NOT. That is the entire reason<br/>three separate families exist."]
  A2 -.-> NOTE
  A3 -.-> NOTE
  A5 -.-> NOTE

The barrier families organised by the observer each one orders against. What it shows: the kernel does not have “a barrier” — it has six families, and the family is chosen by who else is looking at this memory, not by how strong you feel the ordering needs to be. The insight to take: the single most common category error in driver code is reaching for smp_wmb() when the other party is a device rather than a CPU. On x86 that mistake is invisible (smp_wmb() and dma_wmb() both compile to nothing there); on arm64 it is a real bug, because smp_wmb() is dmb(ishst) — inner-shareable, the CPUs — while dma_wmb() is dmb(oshst) — outer-shareable, which includes the device. Same source, same architecture, different shareability domain, silent data corruption.

The master table

Macro	Family	What it orders	On a UP (`!CONFIG_SMP`) build	Where defined
`barrier()`	compiler	nothing at the hardware level; forbids the compiler moving any access across it	unchanged — always a compiler fence	`include/linux/compiler.h`: `__asm__ __volatile__("": : :"memory")`
`READ_ONCE(x)` / `WRITE_ONCE(x, v)`	compiler + same-variable	forces exactly one real load/store; carries the implicit address-dependency barrier	unchanged	`include/asm-generic/rwonce.h`
`mb()` / `rmb()` / `wmb()`	mandatory	all/loads/stores, against every observer including devices	still emits an instruction	arch header, wrapped by `asm-generic`
`smp_mb()` / `smp_rmb()` / `smp_wmb()`	SMP-conditional	all/loads/stores, against other CPUs	collapses to `barrier()`	`asm-generic/barrier.h`
`smp_store_release(p, v)` / `smp_load_acquire(p)`	SMP-conditional, one-way	prior accesses before the store / the load before subsequent accesses	collapses to `barrier()` + the access	`asm-generic/barrier.h`, arch-overridden
`smp_store_mb(var, value)`	SMP-conditional	a `WRITE_ONCE` followed by a full barrier	`WRITE_ONCE` + `barrier()`	`asm-generic/barrier.h`
`smp_mb__before_atomic()` / `smp_mb__after_atomic()`	SMP-conditional	upgrades an unordered RMW (`atomic_inc`, `set_bit`) to full ordering on one side	`barrier()`	`asm-generic/barrier.h`
`smp_acquire__after_ctrl_dep()`	SMP-conditional	adds the load→load half a control dependency lacks; defined as `smp_rmb()`	`barrier()` (via `smp_rmb`)	`asm-generic/barrier.h`
`smp_cond_load_relaxed(p, cond)` / `smp_cond_load_acquire(p, cond)`	SMP-conditional	spin-wait; the `_acquire` form adds `smp_acquire__after_ctrl_dep()`	as above	`asm-generic/barrier.h`, arm64/RISC-V override
`smp_mb__after_spinlock()`	SMP-conditional	upgrades a lock acquisition from RCpc to RCsc	`kcsan_mb()` → nothing	`include/linux/spinlock.h`
`smp_mb__after_unlock_lock()`	SMP-conditional	restores transitivity across an unlock-then-lock of different locks	nothing	`include/linux/rcupdate.h`, gated on `CONFIG_ARCH_WEAK_RELEASE_ACQUIRE`
`smp_mb__after_switch_mm()`	SMP-conditional	the barrier `switch_mm()` may already imply	—	`asm-generic/barrier.h`, x86 overrides to nothing
`dma_mb()` / `dma_rmb()` / `dma_wmb()`	DMA	accesses to consistent memory shared with a bus-mastering device	still emits an instruction	`asm-generic/barrier.h`, arch-overridden
`virt_mb()` / `virt_rmb()` / `virt_wmb()` / `virt_store_mb()` / `virt_store_release()` / `virt_load_acquire()` / `virt_mb__before_atomic()` / `virt_mb__after_atomic()`	virtualisation	a possibly-SMP host	identical code on SMP and UP	`asm-generic/barrier.h`
`pmem_wmb()`	persistence	stores have reached a platform durability domain	emits an instruction	`asm-generic/barrier.h` (defaults to `wmb()`); PowerPC overrides
`io_stop_wc()`	write-combining	stops merging of write-combining accesses across it	—	`asm-generic/barrier.h` (default no-op); arm64 → `DGH`
`barrier_nospec()`	speculation	stops speculative execution past the point, not architectural reordering	—	`include/linux/nospec.h` (default no-op); x86 → `LFENCE`
`array_index_nospec(i, sz)`	speculation	masks an index to `[0, sz)` in a way speculation cannot bypass	—	`include/linux/nospec.h`

The v6.12 barrier inventory. What it shows: nineteen rows, of which only three (the smp_* rows) are what most people mean by “a memory barrier”. The insight to take: read the third column top to bottom. Only the SMP-conditional family disappears on a uniprocessor build — and that is the whole design. memory-barriers.txt states the rationale: “SMP memory barriers are reduced to compiler barriers on uniprocessor compiled systems because it is assumed that a CPU will appear to be self-consistent.” A device is not a CPU and cannot be assumed self-consistent with you, so dma_*() and mb() survive. A hypervisor host is also not your CPU, so virt_*() survives. If you find yourself asking “does this barrier still do anything on a UP kernel?”, you are really asking “is the other party a CPU?”.

Two rows deserve an immediate warning, because they look like barriers and are not memory barriers at all. barrier_nospec() and array_index_nospec() are speculation barriers, added in the Spectre response of 2018 (include/linux/nospec.h carries 2018 copyrights from Linus Torvalds, Alexei Starovoitov and Intel). They constrain what the CPU may do speculatively — that is, transiently, before an instruction retires — and they exist to stop a mis-speculated array access from leaving a cache-timing footprint. They have nothing to do with the order in which architecturally-visible accesses become visible to other CPUs. Using barrier_nospec() where you meant smp_rmb() orders nothing; using smp_rmb() where you meant array_index_nospec() mitigates nothing. The array_index_nospec() comment states the requirement that makes it different from an ordinary bounds check: “Always calculate and emit the mask even if the compiler thinks the mask is not needed. The compiler does not take into account the value of @index under speculation.”

The Four-Layer Macro Dispatch

The single most confusing thing about reading arch/*/include/asm/barrier.h for the first time is the underscores. Why does x86 define __smp_mb() and arm64 define __smp_mb() but the code you write says smp_mb()? And why does x86-64 define __mb() while x86-32 defines mb() directly? There is a precise, four-layer scheme, and once you see it every arch header reads the same way.

flowchart TB
  ARCH["<b>Layer 1 — arch private</b><br/>arch/&lt;arch&gt;/include/asm/barrier.h defines<br/>__mb() __rmb() __wmb()<br/>__dma_mb() __dma_rmb() __dma_wmb()<br/>__smp_mb() __smp_rmb() __smp_wmb()<br/>__smp_store_release() __smp_load_acquire()<br/><i>raw instructions, no instrumentation</i>"]
  GEN["<b>Layer 2 — generic fallback</b><br/>include/asm-generic/barrier.h fills every<br/>hole with the next-strongest thing:<br/>rmb() defaults to mb();<br/>dma_rmb() defaults to rmb();<br/>__smp_mb() defaults to mb();<br/>__smp_store_release() defaults to<br/>__smp_mb() + WRITE_ONCE"]
  KCSAN["<b>Layer 3 — KCSAN instrumentation</b><br/>mb() = do { kcsan_mb(); __mb(); } while (0)<br/>Same for rmb/wmb/dma_*/smp_*.<br/>kcsan_mb() is do { } while (0) unless<br/>CONFIG_KCSAN_WEAK_MEMORY"]
  SMP{"<b>Layer 4 — CONFIG_SMP</b>"}
  YES["CONFIG_SMP=y<br/>smp_mb() = kcsan_mb(); __smp_mb()"]
  NO["CONFIG_SMP=n<br/>smp_mb() = barrier()<br/><i>the instruction vanishes</i>"]
  VIRT["<b>virt_* bypasses layer 4</b><br/>virt_mb() = kcsan_mb(); __smp_mb()<br/>— unconditionally, SMP or not"]
  ARCH --> GEN --> KCSAN --> SMP
  SMP -->|"yes"| YES
  SMP -->|"no"| NO
  KCSAN --> VIRT
  MAND["Mandatory mb()/rmb()/wmb() and<br/>dma_*() never reach layer 4 —<br/>they are unconditional by construction"]
  KCSAN --> MAND

How a barrier macro is resolved, in four layers. What it shows: the double-underscore names are the uninstrumented, unconditional arch primitives; the public names are those wrapped in KCSAN instrumentation and, for the smp_* family only, in a CONFIG_SMP test. The insight to take: this is why virt_mb() exists and why its definition is a single line — #define virt_mb() do { kcsan_mb(); __smp_mb(); } while (0) reaches straight past layer 4 to the arch primitive, so it emits the same instruction whether or not the guest kernel was built with SMP support. memory-barriers.txt describes exactly that behaviour: the virt_* macros “have the same effect as smp_mb() etc when SMP is enabled, but generate identical code for SMP and non-SMP systems.”

The generic fallback chain (layer 2) is worth reading literally, because it explains several arch behaviours that otherwise look like bugs:

#ifndef mb
#define mb()	barrier()      /* an arch with no memory barriers at all */
#endif
#ifndef rmb
#define rmb()	mb()           /* no read barrier?  use the full one     */
#endif
#ifndef wmb
#define wmb()	mb()
#endif
#ifndef dma_mb
#define dma_mb()	mb()       /* no DMA barrier?  use the mandatory one */
#endif
#ifndef dma_rmb
#define dma_rmb()	rmb()
#endif
#ifndef dma_wmb
#define dma_wmb()	wmb()
#endif
#ifndef __smp_mb
#define __smp_mb()	mb()       /* no SMP barrier?  use the mandatory one */
#endif

Line by line: each fallback substitutes something at least as strong, never something weaker, so a new port that defines only __mb() is correct — merely slow. This is why PowerPC, which defines __dma_rmb() and __dma_wmb() but no __dma_mb(), still has a working dma_mb(): it silently becomes mb(), which on PowerPC is the heavyweight sync. And it is why smp_store_release() on an arch with no override is __smp_mb(); WRITE_ONCE(...) — a full barrier doing a half barrier’s job, correct but pessimal.

The KCSAN layer (layer 3) is not decoration either. The Kernel Concurrency Sanitizer, merged in 5.8, is a dynamic data-race detector; with CONFIG_KCSAN_WEAK_MEMORY it also models weak memory by delaying accesses in an instrumented build, and it needs to know where the barriers are in order to know which delays are legal. include/linux/kcsan-checks.h (v6.12) shows the three states each kcsan_mb() can take: a real __kcsan_mb() call when barrier instrumentation is on, a compiler __atomic_signal_fence() when KCSAN’s own runtime is being compiled, and do { } while (0) — nothing at all — in an ordinary production kernel. So the wrapper costs a production build exactly zero, which is why the kernel was willing to put it on every barrier macro in the tree.

The Four Basic Barriers and Two Implicit Varieties

Documentation/memory-barriers.txt enumerates memory barriers in four basic varieties plus two implicit ones; the kernel implements them as the macros below. Its “CPU MEMORY BARRIERS” section then prints the canonical grid — reproduced here verbatim, because it is the compressed statement of the whole mandatory-versus-SMP distinction:

	TYPE			MANDATORY	SMP CONDITIONAL
	=======================	===============	===============
	GENERAL			mb()		smp_mb()
	WRITE			wmb()		smp_wmb()
	READ			rmb()		smp_rmb()
	ADDRESS DEPENDENCY			READ_ONCE()

“The Linux kernel has seven basic CPU memory barriers”, the document says — three mandatory, three SMP-conditional, and the address-dependency barrier that has no mandatory form because it is not a barrier instruction at all, only a marked load. The document adds a rule that reads as pedantry and is in fact the commonest review comment in driver code: “Mandatory barriers should not be used to control SMP effects, since mandatory barriers impose unnecessary overhead on both SMP and UP systems. They may, however, be used to control MMIO effects on accesses through relaxed memory I/O windows.”

There are exactly four ways two accesses can be ordered — load-then-load, load-then-store, store-then-store, store-then-load — and each barrier is defined by which of the four it forbids reordering:

flowchart TB
  subgraph PAIRS["The four reorderable pairs"]
    direction TB
    LL["LOAD → LOAD"]
    LS["LOAD → STORE"]
    SS["STORE → STORE"]
    SL["STORE → LOAD<br/><i>the one even x86-TSO performs</i>"]
  end
  RMB["<b>smp_rmb()</b><br/>forbids LOAD→LOAD only"]
  WMB["<b>smp_wmb()</b><br/>forbids STORE→STORE only"]
  ACQ["<b>acquire load</b><br/>forbids LOAD→LOAD and LOAD→STORE<br/>= strictly stronger than smp_rmb()"]
  REL["<b>release store</b><br/>forbids LOAD→STORE and STORE→STORE"]
  MB["<b>smp_mb()</b><br/>forbids ALL FOUR — the only one<br/>that touches STORE→LOAD"]
  DEP["<b>address dependency</b><br/>(READ_ONCE / rcu_dereference)<br/>forbids LOAD→LOAD only for the<br/><i>dependent</i> load. Zero instructions."]
  LL --- RMB
  SS --- WMB
  LL --- ACQ
  LS --- ACQ
  LS --- REL
  SS --- REL
  SL --- MB
  LL --- DEP

The four reorderable access pairs and which primitive forbids each. What it shows: every barrier in the catalogue is a subset of the same four-cell space, and the subsets overlap in a specific way — an acquire load is strictly stronger than smp_rmb() because it also covers LOAD→STORE. The insight to take: the STORE → LOAD cell has exactly one occupant. No directional barrier, no acquire, no release, and no dependency orders a store followed by a load; only smp_mb() does. That is why smp_mb() is the only SMP barrier that costs a real instruction on x86, and it is the single fact that decides most “do I need a full barrier here?” arguments. The canonical STORE→LOAD cases in the kernel are the seqlock writer, Dekker-style mutual exclusion, and the sleeper’s set_current_state()-then-test-the-flag sequence — which is precisely why set_current_state() is built on smp_store_mb() (a store followed by a full barrier) rather than on a release store.

A write (store) barrier, smp_wmb(), “gives a guarantee that all the STORE operations specified before the barrier will appear to happen before all the STORE operations specified after the barrier with respect to the other components of the system.” It is a partial ordering on stores only — it says nothing about loads. The doc notes that a write barrier should “normally be paired with read or address-dependency barriers.”

A read (load) barrier, smp_rmb(), is “a partial ordering on loads only” — all loads before it appear to complete before all loads after it. It is strictly stronger than an address-dependency barrier and can substitute for one. It does nothing for stores.

A general (full) barrier, smp_mb(), orders both loads and stores: every load and store before it appears to happen before every load and store after it, as seen by other CPUs. A general barrier implies both a read and a write barrier and can substitute for either. It is the only barrier that orders a store followed by a load (the one reordering even strongly-ordered x86 permits — see Cache Coherence and the Store Buffer).

The fourth listed variety, the address-dependency barrier, is marked HISTORICAL in the v6.12 doc: “Kernel release v5.9 removed kernel APIs for explicit address-dependency barriers.” Its semantics are now folded into READ_ONCE() (covered under Dependency Ordering below).

The two implicit varieties are ACQUIRE and RELEASE operations, exposed as smp_load_acquire() and smp_store_release().

Acquire and Release: One-Way Permeable Barriers

The acquire/release pair is subtler and usually cheaper than a full barrier. What follows is the catalogue entry — enough to use the macros and read the lowering table. The full contract (why they compose, what a reads-from match is, why a release beats smp_wmb() across three CPUs, and the RCpc/RCsc distinction) is developed at length in Acquire Release and Fence Semantics and is not repeated here. The doc defines an acquire as “a one-way permeable barrier”: memory operations after the acquire cannot float up above it, but operations before it may sink down past it. Symmetrically, a release guarantees that operations before it cannot sink down past it, but operations after it may float up above it. Picture a release as a trapdoor that lets things fall out of the critical section from above but never in from below; an acquire is the mirror trapdoor at the bottom.

smp_store_release(p, v) publishes a value with release semantics; smp_load_acquire(p) reads it with acquire semantics. The doc states the key transitivity guarantee: “after an ACQUIRE on a given variable, all memory accesses preceding any prior RELEASE on that same variable are guaranteed to be visible.” So if CPU 1 does smp_store_release(&flag, 1) after writing data, and CPU 2 does smp_load_acquire(&flag) and sees 1, then CPU 2 is guaranteed to see all the data CPU 1 wrote before the release. This is the publish-subscribe pattern again, expressed as a single self-documenting pair instead of two separate barriers. Note the doc’s warning: a RELEASE+ACQUIRE pair is not a full barrier — an access before the acquire can still be reordered after a following release.

The deeper semantics — what “acquire”, “release”, and “fence” mean as ordering relations, and how they compose — are developed in Acquire Release and Fence Semantics; the formal underpinning is The Linux Kernel Memory Model.

How the Macros Compile: Same Source, Different Metal

The defining feature of the Linux barrier API is that the same C macro lowers to wildly different machine code depending on the architecture’s memory model. This is where the abstraction earns its keep. The generic fallbacks live in include/asm-generic/barrier.h; each architecture overrides what it must.

This table is the centrepiece of the note. Every cell was read from the v6.12 arch header named in the column heading; a cell marked (generic) means that architecture supplies no override and the asm-generic fallback chain applies, and the resulting expansion is given.

Macro	x86-64 `arch/x86/…/barrier.h`	arm64 `arch/arm64/…/barrier.h`	PowerPC `arch/powerpc/…/barrier.h`	RISC-V `arch/riscv/…/barrier.h`	s390 `arch/s390/…/barrier.h`
`mb()` (mandatory)	`MFENCE`	`DSB SY`	`sync`	`fence iorw,iorw`	`BCR 14,0`
`rmb()` (mandatory)	`LFENCE`	`DSB LD`	`sync`	`fence ir,ir`	`barrier()` — none
`wmb()` (mandatory)	`SFENCE`	`DSB ST`	`sync`	`fence ow,ow`	`barrier()` — none
`dma_mb()`	(generic → `mb()`) `MFENCE`	`DMB OSH`	(generic → `mb()`) `sync`	(generic → `mb()`) `fence iorw,iorw`	(generic → `mb()`) `BCR 14,0`
`dma_rmb()`	`barrier()` — none	`DMB OSHLD`	`lwsync`	(generic → `rmb()`) `fence ir,ir`	`BCR 14,0`
`dma_wmb()`	`barrier()` — none	`DMB OSHST`	`lwsync` / `mbar` / `eieio` by sub-arch	(generic → `wmb()`) `fence ow,ow`	`BCR 14,0`
`smp_mb()`	`LOCK; ADDL $0,-4(%rsp)`	`DMB ISH`	`sync`	`fence rw,rw`	`BCR 14,0`
`smp_rmb()`	`barrier()` — none	`DMB ISHLD`	`lwsync`	`fence r,r`	`barrier()` — none
`smp_wmb()`	`barrier()` — none	`DMB ISHST`	`lwsync` / `mbar` / `eieio`	`fence w,w`	`barrier()` — none
`smp_store_release(p,v)`	`barrier()` + `MOV`	`STLRB`/`STLRH`/`STLR` by size	`lwsync` + store	`fence rw,w` + store	`barrier()` + store
`smp_load_acquire(p)`	`MOV` + `barrier()`	`LDARB`/`LDARH`/`LDAR` by size	load + `lwsync`	load + `fence r,rw`	load + `barrier()`
`smp_store_mb(var,v)`	`XCHG` (a locked RMW)	(generic) `WRITE_ONCE` + `DMB ISH`	(generic) store + `sync`	(generic) store + `fence rw,rw`	(generic) store + `BCR 14,0`
`smp_mb__before_atomic()` / `__after_atomic()`	`do { } while (0)` — free	(generic → `smp_mb()`) `DMB ISH`	(generic) `sync`	(generic) `fence rw,rw`	`barrier()` — none
`smp_mb__after_spinlock()`	(generic) `kcsan_mb()` — free	`smp_mb()` → `DMB ISH`	(generic) free	`fence iorw,iorw`	(generic) free
`smp_mb__after_unlock_lock()`	`do { } while (0)`	`do { } while (0)`	`smp_mb()` → `sync`	`do { } while (0)`	`do { } while (0)`
`smp_mb__after_switch_mm()`	`do { } while (0)` — CR3 write is a barrier	(generic → `smp_mb()`)	(generic)	(generic)	(generic)
`pmem_wmb()`	(generic → `wmb()`) `SFENCE`	(generic → `wmb()`) `DSB ST`	`PHWSYNC`	(generic → `wmb()`) `fence ow,ow`	(generic → `wmb()`) none
`io_stop_wc()`	(generic) no-op	`DGH` (`hint #6`)	(generic) no-op	(generic) no-op	(generic) no-op
`barrier_nospec()`	`LFENCE` (via `ALTERNATIVE` on `X86_FEATURE_LFENCE_RDTSC`)	(uses `CSDB` inside `array_index_mask_nospec`)	fixup-section `NOSPEC_BARRIER_SLOT` under `CONFIG_PPC_BARRIER_NOSPEC`	(generic) no-op	(uses `array_index_mask_nospec`)

What every barrier macro compiles to on five architectures, Linux 6.12. What it shows: the same nineteen source lines produce five completely different instruction sequences, and the columns are not simply “strong” and “weak” — each architecture has its own idiosyncratic shape. The insight to take: four cells are worth staring at. (1) x86 dma_rmb()/dma_wmb() are nothing at all, so the commonest driver barrier bug — reaching for an SMP barrier where a DMA barrier belongs, or omitting it entirely — is completely invisible on x86 and is a DMB OSHLD versus DMB ISHLD shareability mismatch on arm64. (2) PowerPC’s mandatory rmb() and wmb() are both the full sync, not something lighter; there is no cheap mandatory read barrier on POWER, which is why driver hot paths there reach for dma_rmb() (a lwsync) instead. (3) s390 inverts the usual hierarchy: rmb() and wmb() emit no instruction while dma_rmb() and dma_wmb() are a full BCR serialisation — the DMA barriers are strictly stronger than the mandatory ones of the same direction. (4) PowerPC is the only architecture where smp_mb__after_unlock_lock() costs anything, because it is the only one that selects CONFIG_ARCH_WEAK_RELEASE_ACQUIRE (verified by grepping arch/*/Kconfig at v6.12).

Uncertain

Verify: why s390 defines __dma_rmb()/__dma_wmb() as the full __mb() serialisation while defining __rmb()/__wmb() as bare barrier(). Reason: the code is unambiguous — arch/s390/include/asm/barrier.h v6.12 reads #define __rmb() barrier() / #define __wmb() barrier() / #define __dma_rmb() __mb() / #define __dma_wmb() __mb() — but the header carries no comment explaining the asymmetry, and it inverts the strength ordering that memory-barriers.txt implies (mandatory ≥ DMA ≥ SMP). The plausible reading is conservatism about I/O ordering on z Systems, where the mandatory barriers are considered unnecessary for CPU-visible ordering but a device handshake is not modelled the same way. To resolve: read the commit history for arch/s390/include/asm/barrier.h and the s390 architecture principles of operation on the ordering of accesses to I/O-attached storage. Do not rely on the reason; the code above is verified. uncertain

x86: Strong Ordering Makes SMP Barriers Nearly Free

x86 is a Total Store Order (TSO) machine: the only reordering it performs is letting a load overtake an earlier store to a different address. It never reorders load-load, store-store, or store-after-load-of-different-addr. So on x86, smp_rmb() and smp_wmb() need no actual fence instruction — only a compiler barrier to stop GCC from reordering. From arch/x86/include/asm/barrier.h (v6.12):

#define __smp_rmb()	dma_rmb()        /* dma_rmb() == barrier() on x86 */
#define __smp_wmb()	barrier()        /* compiler barrier only */
#define __smp_mb()	asm volatile("lock; addl $0,-4(%%" _ASM_SP ")" ::: "memory", "cc")

__smp_wmb() is literally barrier() — a compiler directive (asm volatile("" ::: "memory")), emitting zero instructions. TSO already orders store-store.
__smp_rmb() resolves to dma_rmb(), which on x86 is also barrier() — TSO orders load-load.
__smp_mb() is the one that costs something. It is a lock-prefixed dummy addl $0 to the top of the stack. The lock prefix is a full fence that drains the store buffer, ordering the store-before-load case TSO otherwise allows. (mfence would also work and is used for the mandatory __mb() via ALTERNATIVE, but a locked instruction is cheaper on many microarchitectures and avoids mfence’s broader semantics.)

The acquire/release macros on x86 are equally lean — just a compiler barrier wrapped around a READ_ONCE/WRITE_ONCE:

#define __smp_store_release(p, v)					\
do {									\
	compiletime_assert_atomic_type(*p);				\
	barrier();							\
	WRITE_ONCE(*p, v);						\
} while (0)

The barrier() is a pure compiler fence; no CPU instruction is needed because TSO already gives the store release ordering for free.

ARM64: Weak Ordering Needs Real Fences

AArch64 is a weakly-ordered architecture: absent barriers, it may reorder almost any pair of accesses to different locations. So the identical C macros must emit genuine fence instructions. From arch/arm64/include/asm/barrier.h (v6.12):

#define __smp_mb()	dmb(ish)
#define __smp_rmb()	dmb(ishld)
#define __smp_wmb()	dmb(ishst)

dmb(ish) is a Data Memory Barrier in the inner-shareable domain (the cores that share coherent memory) — a full barrier.
dmb(ishld) orders prior loads against later loads and stores (the read barrier).
dmb(ishst) orders prior stores against later stores (the write barrier).

And the acquire/release macros use ARMv8’s dedicated load-acquire / store-release instructions rather than a separate fence:

case 8:								\
	asm volatile ("stlr %x1, %0" ... );	/* store-release */
	...
	asm volatile ("ldar %0, %1" ... );	/* load-acquire  */

stlr (store-release register) and ldar (load-acquire register) bake the one-way ordering into the memory instruction itself — strictly cheaper than dmb + plain store, because they only constrain ordering relative to this access, not a blanket fence. This is the architectural payoff of having acquire/release in the API: on weak machines the compiler can pick the cheap instruction; on x86 it costs nothing extra anyway.

The takeaway: write smp_wmb()/smp_rmb()/smp_load_acquire() everywhere and let the arch layer decide the cost. Code that “works on x86” because the barriers happen to be no-ops there is a latent bug on ARM64 — the missing-barrier failure is invisible on x86 and reproducible on ARM.

The Publish-Then-Flag Pattern, Step by Step

One worked example belongs here — the one that shows why the catalogue has pairs — and the rest of the litmus-test material belongs to Acquire Release and Fence Semantics, which develops MP, SB, LB, WRC, ISA2 and Z6.0 as two-column timelines and derives the choice of primitive from the shape of the cycle. Do not go looking for that analysis here; go there.

The most common barrier use is publishing a data structure and then setting a flag to announce it. McKenney’s “Memory Barriers: a Hardware View for Software Hackers” gives the canonical worked example (McKenney 2010):

void foo(void)            /* writer, on CPU 0 */
{
	a = 1;            /* the data */
	smp_wmb();        /* write barrier */
	b = 1;            /* the "ready" flag */
}
 
void bar(void)            /* reader, on CPU 1 */
{
	while (b == 0)
		continue;  /* spin until flag set */
	smp_rmb();         /* read barrier */
	assert(a == 1);    /* must hold */
}

Walk it through on a weakly-ordered machine. Without smp_wmb() in foo(), CPU 0’s store to b can become visible to CPU 1 before its store to a (the hardware reason — store buffers — is in Cache Coherence and the Store Buffer). CPU 1 then sees b==1, exits the loop, and reads the stale a==0: the assertion fires. The smp_wmb() forces a=1 to be committed before b=1 becomes visible.

But the writer barrier alone is insufficient. Without smp_rmb() in bar(), CPU 1 may speculatively load a early — before it even loads b — caching the old value, then later load b, see 1, and use the stale a. The doc’s “Read memory barriers vs load speculation” section explains exactly this: CPUs prefetch loads when the bus is idle. The smp_rmb() forces the load of a to be re-executed (or its speculation discarded) after b is seen. Both barriers are mandatory; each fixes a reordering on its own side, and only together do they establish cross-CPU order.

A cleaner modern idiom collapses the pair into acquire/release:

/* writer */                       /* reader */
a = 1;                             if (smp_load_acquire(&b))
smp_store_release(&b, 1);              assert(a == 1);

Here smp_store_release(&b, 1) guarantees a=1 precedes the publication of b, and smp_load_acquire(&b) guarantees that if b reads as 1, the subsequent load of a sees the published value. Self-documenting, and on ARM64 it compiles to a single stlr/ldar pair.

Dependency Ordering and Why READ_ONCE Is Mandatory

There is a category of ordering the hardware gives you for free, called address (data) dependency ordering. If the second load’s address is computed from the first load’s value — the classic “load a pointer, then dereference it” — then essentially every CPU Linux supports orders those two loads automatically, because the CPU literally cannot fetch *p until it knows the value of p. The doc’s example:

CPU 1                  CPU 2
B = 4;
<write barrier>
WRITE_ONCE(P, &B);     Q = READ_ONCE(P);
                       <implicit address-dependency barrier>
                       D = *Q;

Because the load of *Q depends on the value loaded into Q, the read side needs no explicit smp_rmb() — the dependency suffices. This is exactly the mechanism that makes RCU readers nearly free: see Read-Copy-Update Fundamentals and The publish-subscribe Pattern in RCU.

There are two essential caveats. First, the DEC Alpha exception: Alpha had split caches that could reorder even dependent loads, observing (Q == &B) and (D == 2) — the old value of B through a freshly-published pointer. As the doc records, “As of v4.15 of the Linux kernel, an smp_mb() was added to READ_ONCE() for DEC Alpha,” so today the dependency barrier is implicit in READ_ONCE() on every architecture and explicit smp_read_barrier_depends() was removed.

Second — and this is the live, every-architecture trap — the dependency must survive the compiler, and a plain C load is not guaranteed to preserve it. This is why READ_ONCE() is mandatory for dependency-ordered loads. The doc’s “CPU MEMORY BARRIERS” table lists the address-dependency barrier’s API as simply READ_ONCE(), and warns that with a plain load “there is no guarantee in the C specification that the compiler may not speculate the value of b … and load a[b] before b,” or reload b after loading a[b], ending up with a newer b than the a[b] it indexed. If the compiler proves (or guesses) the pointer’s value and hoists the dependent load, the address dependency that the hardware was relying on simply vanishes from the emitted code, and ordering is lost on every CPU. READ_ONCE() forces a single real load through the variable, preserving the dependency the silicon needs. The dirty details of how compilers break dependencies live in Documentation/RCU/rcu_dereference.rst; the mechanics of READ_ONCE/WRITE_ONCE themselves are in Compiler Barriers and READ_ONCE WRITE_ONCE.

A separate, weaker cousin is the control dependency (the second access is guarded by an if on the first load rather than addressed by its value). The doc is emphatic that control dependencies are fragile: “A load-load control dependency requires a full read memory barrier, not simply an (implicit) address-dependency barrier” — compilers “do not understand them” and will happily hoist a load out of both branches of an if. Control dependencies order load→store reliably (the store cannot be speculated before the branch resolves) but not load→load, so do not lean on them for read ordering.

Mandatory and DMA Barriers: Talking to Devices

The SMP barriers above evaporate to compiler barriers on a uniprocessor build, “because it is assumed that a CPU will appear to be self-consistent.” But ordering also matters when a CPU talks to a device, and a device is not a CPU — it is not party to the cache-coherence protocol and SMP barriers do not constrain it. The kernel therefore provides a parallel family of mandatory barriers that emit a real fence even on UP, and DMA barriers for coherent memory shared with a bus-mastering device.

This distinction — dma_*mb() versus smp_*mb() — is routinely confused, and the confusion is invisible on the architecture most people develop on. It is worth drawing, because the difference is a scope difference, not a strength difference:

flowchart TB
  subgraph CPUS["Inner-shareable domain — the CPUs"]
    direction LR
    C0["CPU 0"]
    C1["CPU 1"]
    C2["CPU 2"]
  end
  subgraph OUTER["Outer-shareable domain — CPUs PLUS coherent bus masters"]
    direction LR
    DMA["DMA engine / NIC / GPU<br/>reads dma_alloc_coherent memory<br/><i>coherent, but not a CPU</i>"]
  end
  MEM[("Consistent (coherent) memory<br/>e.g. a descriptor ring")]
  MMIO[("MMIO register window<br/><i>outside the cache system entirely</i>")]
  C0 --- MEM
  DMA --- MEM
  C0 -.->|"doorbell write"| MMIO
  DMA -.->|"reads its own registers"| MMIO
  B1["<b>smp_wmb()</b> — arm64 DMB ISHST<br/>orders my stores as seen by <b>other CPUs</b>.<br/>Says NOTHING about the DMA engine."]
  B2["<b>dma_wmb()</b> — arm64 DMB OSHST<br/>orders my stores as seen by a<br/><b>coherent device</b> in the outer domain."]
  B3["<b>writel() / mb()</b><br/>the ONLY things that order an<br/><b>MMIO</b> access. dma_*() does not."]
  CPUS -.-> B1
  OUTER -.-> B2
  MMIO -.-> B3

The three visibility domains and the barrier that fences each. What it shows: smp_*mb(), dma_*mb() and the MMIO accessors are not three strengths of the same thing — they are three different audiences. The inner-shareable domain is the CPUs; the outer-shareable domain adds coherent bus masters; MMIO is not in any coherence domain at all. The insight to take: the arm64 lowering makes the scope difference literal — smp_wmb() is DMB ISHST and dma_wmb() is DMB OSHST, the same barrier type with a wider domain. Writing smp_wmb() before handing a descriptor to a device is therefore not “a slightly weak barrier”; it is a barrier aimed at the wrong audience, and the device may see the ownership flag before the payload. On x86 both compile to nothing, so the mistake ships. And the third arrow is the one people get wrong even after learning the first two: memory-barriers.txt states flatly that “the dma_*() barriers do not provide any ordering guarantees for accesses to MMIO regions.”

The mb() / rmb() / wmb() mandatory barriers are full/read/write fences that always emit a hardware instruction, on SMP and UP builds alike. Read from v6.12: on x86-64 they are plain mfence / lfence / sfence (#define __mb() asm volatile("mfence":::"memory") and so on) — note that the ALTERNATIVE(...)-guarded forms that select between lock; addl and the SSE2 fences exist only in the CONFIG_X86_32 branch of the same header, where X86_FEATURE_XMM2 may be absent; on 64-bit, SSE2 is architectural and the fences are unconditional. On arm64 they are dsb(sy) / dsb(ld) / dsb(st) — dsb (Data Synchronization Barrier) is strictly heavier than the dmb (Data Memory Barrier) used for SMP barriers, because dsb waits for completion rather than merely ordering, which is what a device handshake needs. On PowerPC all three are the same heavyweight sync. The doc cautions: “Mandatory barriers should not be used to control SMP effects, since mandatory barriers impose unnecessary overhead on both SMP and UP systems. They may, however, be used to control MMIO effects.”

The DMA barriers dma_rmb() / dma_wmb() / dma_mb() order accesses to consistent (coherent) memory shared between the CPU and a DMA-capable device — for example a ring of descriptors. The doc’s worked driver example:

if (desc->status != DEVICE_OWN) {
	dma_rmb();                  /* don't read data until we own the descriptor */
	read_data = desc->data;
	desc->data = write_data;
	dma_wmb();                  /* flush modifications before status update */
	desc->status = DEVICE_OWN;  /* hand ownership to the device */
	writel(DESC_NOTIFY, doorbell);
}

The dma_rmb() ensures the CPU does not read desc->data before it has confirmed it owns the descriptor; the dma_wmb() ensures the data writes land before the status field flips ownership to the device. On x86 these are again just compiler barriers (__dma_rmb() and __dma_wmb() both expand to barrier()), because TSO already orders same-direction accesses; on ARM64 they are dmb(oshld) / dmb(oshst) — the outer-shareable domain, which includes the device, rather than the inner-shareable domain used by the SMP barriers. Critically, the doc warns the dma_*() barriers do not order MMIO register accesses; for memory-mapped registers you need the I/O accessors (readl/writel) or mb().

The I/O Accessors: Where `readl()` and `writel()` Get Their Ordering

The catalogue would be incomplete without the family most drivers actually use, because in practice the right answer to “which barrier for this MMIO access?” is usually “none — use the accessor.” readX() and writeX() are not thin wrappers around a volatile pointer dereference; on a weakly-ordered architecture they carry barriers inside them, and memory-barriers.txt specifies exactly five ordering properties a portable driver may rely on for a pointer mapped with default I/O attributes (i.e. from ioremap()):

#	Guarantee	What it lets a driver do
1	All `readX()`/`writeX()` to the same peripheral are ordered with respect to each other	Register writes arrive in program order — the address-register-then-data-register idiom works
2	A `writeX()` issued while holding a spinlock is ordered before a `writeX()` from another CPU that later acquires the same lock	MMIO writes arrive in an order consistent with lock acquisitions
3	A `writeX()` waits for all prior writes to memory issued by or propagated to the same thread	Writing a DMA buffer then ringing the doorbell works without an explicit `dma_wmb()`
4	A `readX()` completes before any subsequent read from memory by the same thread	Polling a “DMA complete” status register then reading the buffer will not return stale data
5	A `readX()` completes before any subsequent `delay()` loop	`writel(); readl(); udelay(1); writel();` really does put ≥ 1 µs between the two writes

The five portable I/O ordering guarantees from memory-barriers.txt v6.12. What it shows: guarantees 3 and 4 are the ones that matter for DMA, and they are precisely the dma_wmb() and dma_rmb() you would otherwise have to write yourself. The insight to take: if a driver uses readl()/writel() throughout, it needs neither mandatory nor DMA barriers around its register accesses — that is the entire point of the accessors. The corresponding trap is the _relaxed() forms: readX_relaxed()/writeX_relaxed() keep only guarantee 1 and drop 2 through 5, so a writel_relaxed() doorbell after a coherent-memory descriptor write is a real bug that the non-relaxed form would have prevented. readsX()/writesX() — the FIFO string accessors — carry only the relaxed guarantees too.

Reading the arm64 implementation makes the mechanism concrete, and shows that guarantees 3 and 4 are literally the DMA barriers (arch/arm64/include/asm/io.h, v6.12):

/* IO barriers */
#define __io_ar(v)							\
({									\
	unsigned long tmp;						\
	dma_rmb();							\
	/*								\
	 * Create a dummy control dependency from the IO read to any	\
	 * later instructions. This ensures that a subsequent call to	\
	 * udelay() will be ordered due to the ISB in get_cycles().	\
	 */								\
	asm volatile("eor	%0, %1, %1\n"				\
		     "cbnz	%0, ."					\
		     : "=r" (tmp) : "r" ((unsigned long)(v))		\
		     : "memory");					\
})
 
#define __io_bw()		dma_wmb()
#define __io_br(v)
#define __io_aw(v)
#define __iomb()		dma_mb()

Line by line. __io_bw() — before write — is exactly dma_wmb(), which is guarantee 3. __io_ar(v) — after read — is dma_rmb(), which is guarantee 4, plus a hand-built control dependency: eor %0, %1, %1 exclusive-ORs the loaded value with itself, producing a register that is provably zero but that the CPU cannot know is zero until the load has returned; cbnz %0, . then branches on it. The branch is never taken, but the CPU cannot resolve it before the MMIO read completes, so nothing after it can execute early. That is guarantee 5 — the udelay() case — implemented as a data hazard the hardware must respect rather than as another fence. __io_br() and __io_aw() are empty because arm64 needs nothing there. And __iomb() — used for readX()/writeX() mutual ordering — is dma_mb(), i.e. DMB OSH.

The generic wiring that turns these hooks into the accessors lives in include/linux/io.h and asm-generic/io.h; an architecture supplies the four __io_* hooks and gets all of readb/readw/readl/readq and their write counterparts, correctly barriered, for free. This is the same layering idea as the barrier headers themselves: the arch supplies primitives, the generic layer supplies policy.

`virt_*`: Barriers for a Guest Talking to a Host

The virt_* family is small, easy to miss, and solves a problem that only exists under virtualisation. A guest kernel compiled without SMP support still runs on a host that may be SMP, and the host-side device emulation — a vhost thread, another vCPU — is a genuine second processor. memory-barriers.txt names the situation precisely: “Guests running within virtual machines might be affected by SMP effects even if the guest itself is compiled without SMP support. This is an artifact of interfacing with an SMP host while running an UP kernel.”

The obvious fix — use mandatory mb() — works and is wasteful, because mb() on arm64 is DSB SY and on PowerPC is sync, when all that is needed is CPU-to-CPU ordering. So the kernel defines a third family:

/* Barriers for virtual machine guests when talking to an SMP host */
#define virt_mb() do { kcsan_mb(); __smp_mb(); } while (0)
#define virt_rmb() do { kcsan_rmb(); __smp_rmb(); } while (0)
#define virt_wmb() do { kcsan_wmb(); __smp_wmb(); } while (0)
#define virt_store_mb(var, value) do { kcsan_mb(); __smp_store_mb(var, value); } while (0)
#define virt_mb__before_atomic() do { kcsan_mb(); __smp_mb__before_atomic(); } while (0)
#define virt_mb__after_atomic()	do { kcsan_mb(); __smp_mb__after_atomic(); } while (0)
#define virt_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
#define virt_load_acquire(p) __smp_load_acquire(p)

Every one of them reaches the double-underscore arch primitive directly, skipping the #ifdef CONFIG_SMP test that the public smp_* names go through. That single design decision is the whole family: virt_mb() is the SMP barrier, unconditionally. The doc’s summary — “these have the same effect as smp_mb() etc when SMP is enabled, but generate identical code for SMP and non-SMP systems” — is a direct reading of those macro bodies. Note also the closing caveat, which repeats the domain rule from earlier: the virt_* barriers “do not control MMIO effects: to control MMIO effects, use mandatory barriers.”

Drawn as the exchange it protects:

sequenceDiagram
    autonumber
    participant G as Guest vCPU<br/>(kernel built !CONFIG_SMP)
    participant R as Shared virtqueue<br/>(guest physical memory)
    participant H as Host vhost thread<br/>(a real second CPU)
    Note over G: Guest believes it is uniprocessor.<br/>smp_wmb() has compiled to barrier() —<br/>ZERO instructions.
    G->>R: STORE descriptor payload
    rect rgb(240, 220, 220)
    Note over G: smp_wmb() here would emit nothing.<br/>The two stores may reach the host<br/>in either order.
    end
    G->>R: STORE avail->idx (the "ready" flag)
    H->>R: LOAD avail->idx → sees new index
    H->>R: LOAD descriptor payload → STALE ⚠
    Note over H: Host consumed a descriptor<br/>that was never fully written.
    Note over G,H: FIX: virt_wmb() reaches __smp_wmb() directly,<br/>bypassing the CONFIG_SMP test, so it emits<br/>DMB ISHST on arm64 even in a UP guest.

Why a uniprocessor guest still needs a barrier. What it shows: the guest’s CONFIG_SMP=n build has legitimately compiled smp_wmb() away — the assumption “a CPU is self-consistent with itself” is true — but the host is a second CPU that assumption never accounted for. The insight to take: the bug is not in the barrier’s semantics but in the build-time question #ifdef CONFIG_SMP is asking. It asks “does this kernel run on more than one CPU?”, when the question that matters is “is anyone else looking at this memory?”. virt_wmb() exists precisely because those two questions have different answers under virtualisation, and it answers the second one by skipping the test. Note also that this failure is a build-configuration bug, not an architecture bug: the same guest source built CONFIG_SMP=y would have been correct by accident.

virtio is the canonical consumer, and its header spells out the runtime decision rather than a compile-time one (include/linux/virtio_ring.h, v6.12):

static inline void virtio_wmb(bool weak_barriers)
{
	if (weak_barriers)
		virt_wmb();
	else
		dma_wmb();
}

The header’s own comment explains why this is a runtime branch: “For using virtio to talk to real devices (eg. other heterogeneous CPUs) we do need real barriers. In theory, we could be using both kinds of virtio, so it’s a runtime decision, and the branch is actually quite cheap.” So one driver, one ring layout, two different barrier families selected per device by whether the other end is a hypervisor (weak_barriers = true → virt_wmb()) or actual silicon (weak_barriers = false → dma_wmb()). It is the clearest production illustration in the tree that the barrier family is chosen by who the other party is.

The Specialist Barriers

Five macros in the catalogue exist for one job each, and each one is the answer to a question that the general barriers answer badly.

smp_mb__before_atomic() / smp_mb__after_atomic() — the ordering shim for read-modify-write operations that are atomic but unordered. atomic_inc(), atomic_dec(), set_bit() and clear_bit() are perfectly atomic and order nothing at all; Documentation/atomic_t.txt states the rule in four lines: non-RMW operations are unordered, void-returning RMWs are unordered, value-returning RMWs are fully ordered, and conditional RMWs are unordered on failure. When you need ordering around one of the unordered forms, you could write smp_mb() — but on x86 that is a LOCK ADDL next to an instruction that is already a locked RMW and therefore already a full barrier. Hence the shim, which x86 defines as do { } while (0). The doc’s worked example is the reference-counting pattern:

	obj->dead = 1;
	smp_mb__before_atomic();
	atomic_dec(&obj->ref_count);

“This makes sure that the death mark on the object is perceived to be set before the reference counter is decremented.” One usage rule attaches: do not put unrelated code between the shim and the atomic it augments, because the ordering of that intervening code differs by architecture.

smp_store_mb(var, value) — a WRITE_ONCE() followed by a full barrier, as one macro. The generic definition is exactly that; x86 overrides it to a single XCHG (#define __smp_store_mb(var, value) do { (void)xchg(&var, value); } while (0)), because a locked exchange both stores and fences in one instruction, and is cheaper than MOV plus LOCK ADDL. Its most important user is set_current_state() in the sleep path, discussed below.

smp_acquire__after_ctrl_dep() and smp_cond_load_acquire() — the pair that turns a spin-wait into an acquire without an acquire instruction. A spin loop while (READ_ONCE(*p) != v); already establishes a control dependency, which gives load→store ordering for free; adding smp_rmb() supplies the missing load→load half, and the two together are exactly an acquire. The header states it that way: “A control dependency provides a LOAD→STORE order, the additional RMB provides LOAD→LOAD order, together they provide LOAD→{LOAD,STORE} order, aka. (load)-ACQUIRE.” smp_cond_load_acquire() is built from smp_cond_load_relaxed() plus that macro, and arm64 and RISC-V override smp_cond_load_relaxed() to use WFE/WFET and the Zawrs wrs instruction respectively — so the spin actually sleeps the core rather than burning it. The ordering semantics are the sibling note’s territory; the point here is that this is the macro to reach for whenever you were about to write a spin loop followed by smp_rmb().

pmem_wmb() — persistence, not visibility. Every other barrier in this note answers “who can see my store?”; this one answers “has my store survived?” The doc: it ensures “that stores for which modifications are written to persistent storage reached a platform durability domain,” and is needed “after a non-temporal write to pmem region… in addition to the ordering done by wmb().” The default is wmb(), which is right for x86 (SFENCE after non-temporal stores is the documented persistence sequence there); PowerPC overrides it to PHWSYNC, a persistent-storage-specific heavyweight sync. Loads need nothing extra — “for load from persistent memory, existing read memory barriers are sufficient.”

io_stop_wc() — write-combining. A mapping made with ioremap_wc() lets the CPU merge adjacent stores into larger bus transactions, which is a large win for framebuffers and a latency problem when the merge makes the CPU wait for a neighbouring access that has not arrived. io_stop_wc() splits the merge window. The generic definition is do { } while (0); arm64 implements it as DGH, the Data Gathering Hint (hint #6), whose header comment reads: “This instruction prevents merging memory accesses with Normal-NC or Device-GRE attributes before the hint instruction with any memory accesses appearing after the hint instruction.” Note carefully that this is a performance control, not a correctness one — it changes when accesses are gathered, not the order they are perceived in.

Implicit Barriers: What Already Has One

The last part of the catalogue is the set of things that are barriers without looking like barriers. Reaching for an explicit macro where one of these already applies is the most common form of redundant-barrier review comment.

flowchart TB
  Q["I think I need a barrier here."]
  L{"Is the data protected<br/>by a lock?"}
  L1["<b>Done.</b> spin_lock/mutex_lock is an ACQUIRE,<br/>spin_unlock/mutex_unlock is a RELEASE.<br/>Correctly paired, and lockdep checks it."]
  W{"Am I going to sleep on a<br/>condition, or waking someone?"}
  W1["<b>Done.</b> set_current_state() interpolates a full<br/>barrier via smp_store_mb(); wake_up() executes<br/>a full barrier <i>if it actually wakes something</i>;<br/>wake_up_process() <i>always</i> does."]
  S{"Am I calling schedule()?"}
  S1["<b>Done.</b> schedule() and similar<br/>imply full memory barriers."]
  I{"Am I disabling interrupts<br/>with local_irq_save()?"}
  I1["<b>NOT done.</b> Interrupt disable/enable act as<br/><b>compiler barriers only</b>. If you need memory or<br/>I/O ordering here you must supply it yourself."]
  R["Now reach for an explicit barrier —<br/>and pick its family from the<br/>audience diagram above."]
  Q --> L
  L -->|yes| L1
  L -->|no| W
  W -->|yes| W1
  W -->|no| S
  S -->|yes| S1
  S -->|no| I
  I -->|yes| I1
  I -->|no| R
  I1 --> R

The implicit-barrier checklist, drawn as the order in which to ask the questions. What it shows: four common code shapes already carry ordering, and one — the one that most looks like it should — does not. The insight to take: the local_irq_save() node is the trap. Disabling interrupts feels like taking a lock, and memory-barriers.txt is explicit that it is not one for ordering purposes: “Functions that disable interrupts (ACQUIRE equivalent) and enable interrupts (RELEASE equivalent) will act as compiler barriers only. So if memory or I/O barriers are required in such a situation, they must be provided from some other means.” The doc then works an example where an ethernet driver’s address-register and data-register writes, issued inside an interrupt-disabled section, interleave with the same registers written from its own interrupt handler, producing the execution STORE *ADDR = 3, STORE *ADDR = 4, STORE *DATA = y, q = LOAD *DATA — the data write landing against the wrong address.

The sleep/wake-up case repays a closer look because the barrier is asymmetric and the asymmetry bites. The sleeper’s side is unconditional: set_current_state() is documented as interpolating “a general memory barrier… automatically after it has altered the task state”, and does so via smp_store_mb() — which is why that macro exists. The waker’s side is conditional: “A general memory barrier is executed by wake_up() if it wakes something up. If it doesn’t wake anything up then a memory barrier may or may not be executed; you must not rely on it.” wake_up_process(), by contrast, “always executes a general memory barrier.” The doc’s own litmus-shaped illustration, with X and Y initially zero:

	CPU 1				CPU 2
	===============================	===============================
	X = 1;				Y = 1;
	smp_mb();			wake_up();
	LOAD Y				LOAD X

“If a wakeup does occur, one (at least) of the two loads must see 1. If, on the other hand, a wakeup does not occur, both loads might see 0.” Replace wake_up() with wake_up_process() and the guarantee becomes unconditional.

And there is a second, sharper trap in the same section that no amount of implicit barriers fixes. The barriers implied by sleeper and waker order the task state against the event flag — they do not order the event flag against the data the flag is announcing. The doc’s example: a waker doing my_data = value; event_indicated = 1; wake_up(&q); and a sleeper doing if (event_indicated) do_something(my_data); has “no guarantee that the change to event_indicated will be perceived by the sleeper as coming after the change to my_data.” The fix is the ordinary publish-then-flag pair, added by hand on both sides: smp_wmb() before setting the flag, smp_rmb() after testing it. Implicit barriers order the wake-up protocol; your payload is still your problem.

For completeness, the doc’s own list of implicit barriers: the lock acquisition functions (spinlocks, R/W spinlocks, mutexes, semaphores, R/W semaphores — all ACQUIRE/RELEASE); set_current_state() and its wrappers prepare_to_wait(), prepare_to_wait_exclusive(), and the whole wait_event*() family; the fifteen wake_up*() / complete() waker functions, all providing “the same guarantees of a wake_up() (or stronger)”; and, under “miscellaneous functions”, the single line “schedule() and similar imply full memory barriers.”

Failure Modes and How They Manifest

The defining hazard of memory barriers is that a missing barrier is invisible on the architecture you test on. A driver developed and tested on x86 with smp_wmb() omitted will pass every test, because x86’s TSO orders store-store anyway; the same code panics intermittently on an ARM64 or POWER box under load. This is the single most common real-world barrier bug, and it is why kernel review insists on barriers being present even where they currently compile to nothing.

A second failure mode is the unpaired barrier. Because a barrier only orders the issuing CPU’s accesses, a writer with a perfect smp_wmb() but a reader with no smp_rmb() still races. The doc states it directly: “There is no guarantee that a CPU will see the correct order of effects from a second CPU’s accesses, even if the second CPU uses a memory barrier, unless the first CPU also uses a matching memory barrier.” Reviewers look for the pair; a lone barrier is a code smell.

A third is dropping READ_ONCE()/WRITE_ONCE() around shared accesses (described above): the dependency or even the access itself can be optimized away, fused, or torn by the compiler. This is the failure that the CONFIG_KCSAN (Kernel Concurrency Sanitizer) data-race detector exists to catch at runtime — note the kcsan_mb()/kcsan_rmb()/kcsan_wmb() calls woven into the barrier macros in include/asm-generic/barrier.h, which instrument the barriers so KCSAN can reason about ordering.

A fourth failure mode is unique to the catalogue this note owns: the right barrier from the wrong family. These are not strength errors — the code has a barrier, and it is even the correct direction — it is simply aimed at the wrong audience, and the arch lowering table above is what makes each one diagnosable.

Symptom	Likely mistake	Why it hides	How to spot it
Device reads a descriptor with a valid ownership flag but stale payload; only on arm64/POWER, only under load	`smp_wmb()` where `dma_wmb()` belongs	On x86 both are `barrier()`; on arm64 the difference is `DMB ISHST` vs `DMB OSHST` — inner- vs outer-shareable	Grep the driver for `smp_wmb`/`smp_rmb` near anything reached by `dma_alloc_coherent()`
Correct, but measurably slower than it should be on arm64	`mb()` where `smp_mb()` belongs	Both are correct; `DSB SY` waits for completion, `DMB ISH` only orders	`objdump` the hot path and look for `dsb` where `dmb` would do; `memory-barriers.txt` calls this “unnecessary overhead on both SMP and UP systems”
Ring corruption in a virtio guest built `!CONFIG_SMP`, running on an SMP host	`smp_wmb()` where `virt_wmb()` belongs	On a UP build `smp_wmb()` is literally `barrier()`; the host-side vhost thread is a real second CPU	Any `!CONFIG_SMP` guest driver that talks to a host-emulated device
Doorbell reaches the device before the descriptor payload	`writel_relaxed()` where `writel()` belongs	`_relaxed` keeps only guarantee 1 (same-peripheral ordering) and drops guarantee 3 (prior memory writes complete first)	Search for `_relaxed` accessors on any path that follows a coherent-memory write
Two MMIO writes land closer together than the datasheet’s minimum spacing	`readl_relaxed()` before `udelay()`	Guarantee 5 (the read completes before a `delay()` loop) is a non-relaxed-only property	The `writel(); readl(); udelay(1); writel();` idiom needs the non-relaxed `readl()`
Cache-timing side channel survives a bounds check	`smp_rmb()` where `array_index_nospec()` belongs	A memory barrier orders architectural visibility; it says nothing about speculative execution	Any `if (i < n) { … arr[i] … }` on an attacker-influenced index

Family-mismatch failures and their signatures. What it shows: six distinct bugs that all look like “there is a barrier there, so the ordering must be fine”. The insight to take: every row’s “why it hides” column names either x86 or a UP build. That is not a coincidence — it is the structural reason this whole family distinction exists and the structural reason it is so easy to get wrong. A reviewer’s fastest check is not to reason about the ordering but to ask “who is the other party?” and then confirm the macro’s prefix matches the answer.

A fifth, subtle one is non-multicopy-atomicity. The doc devotes a section to it: most real systems are only “other-multicopy-atomic” — a store becomes visible to all other CPUs simultaneously, but a chain of release-acquire pairs does not guarantee that all CPUs agree on a single global order. Only smp_mb() (a general barrier) restores the global agreement that release-acquire chains lack. If you find yourself reasoning about three or more CPUs and needing them to agree on an order, reach for general barriers, not acquire/release.

When to Choose Which Barrier

Everything in this note collapses into one decision procedure. Two questions decide the family — who is the other party, and does the platform still need help — and only then does a third question decide the strength.

flowchart TB
  START["I have two accesses that<br/>must not be reordered."]
  Q0{"Is the data guarded by a lock<br/>I could just take?"}
  LOCK["<b>Take the lock.</b><br/>Its acquire/release already order this,<br/>lockdep checks the pairing, and<br/>memory-barriers.txt says locking<br/>'is sufficient' for SMP ordering."]
  Q1{"Who is the other party?"}
  DEV_MMIO["<b>An MMIO register</b>"]
  DEV_DMA["<b>A coherent DMA device</b><br/>(descriptor ring, dma_alloc_coherent)"]
  HOST["<b>A hypervisor host</b><br/>(virtio, vhost, paravirt ring)"]
  PMEM["<b>Persistent memory</b>"]
  CPU["<b>Another CPU</b>"]
  A_MMIO["Use readX()/writeX() — they carry<br/>guarantees 1–5. Only if you are on a<br/>relaxed I/O window or using the<br/>_relaxed accessors do you need mb()."]
  A_DMA["dma_rmb() / dma_wmb() / dma_mb()"]
  A_HOST["virt_rmb() / virt_wmb() / virt_mb()"]
  A_PMEM["pmem_wmb(), in addition to wmb()"]
  Q2{"What is the shape<br/>of the ordering?"}
  S1["<b>Publish data, then a flag</b><br/>→ smp_store_release / smp_load_acquire"]
  S2["<b>Dereference a published pointer</b><br/>→ rcu_dereference (READ_ONCE),<br/>address dependency, no fence at all"]
  S3["<b>STORE then LOAD must not swap</b><br/>(Dekker, seqlock writer, wakeup)<br/>→ smp_mb(). Nothing weaker works."]
  S4["<b>Around an unordered atomic</b><br/>(atomic_inc, set_bit)<br/>→ smp_mb__before/after_atomic()"]
  S5["<b>Spin-wait, then use the value</b><br/>→ smp_cond_load_acquire()"]
  S6["<b>3+ CPUs must agree on one order</b><br/>→ smp_mb() throughout;<br/>release-acquire chains are local"]
  START --> Q0
  Q0 -->|yes| LOCK
  Q0 -->|"no — this is a<br/>deliberate lock-free path"| Q1
  Q1 --> DEV_MMIO --> A_MMIO
  Q1 --> DEV_DMA --> A_DMA
  Q1 --> HOST --> A_HOST
  Q1 --> PMEM --> A_PMEM
  Q1 --> CPU --> Q2
  Q2 --> S1 & S2 & S3 & S4 & S5 & S6

Choosing a barrier, as a two-stage decision. What it shows: the audience question comes first and eliminates four of the five branches immediately; only the CPU-to-CPU branch reaches the strength question that most discussions of memory barriers start with. The insight to take: the root node is the one that matters most in review. memory-barriers.txt puts locking above everything — “SMP memory barriers must be used to control the ordering of references to shared memory on SMP systems, though the use of locking instead is sufficient” — and every explicit barrier below it is a deliberate decision to pay attention forever in exchange for a lock’s cost. The second thing to notice is that four of the five audience branches are invisible on x86, where dma_*() and virt_*() and pmem_wmb() all compile to nothing or to a single SFENCE; picking the wrong branch is a bug you cannot reproduce on your laptop.

Spelled out as rules, and largely mechanical once you know what you are ordering:

Publishing data then a flag, one writer / one reader? Prefer smp_store_release() / smp_load_acquire() — self-documenting, cheap on weak arches, and the pairing is obvious to reviewers. Drop to raw smp_wmb()/smp_rmb() only when acquire/release does not fit (e.g. the flag and data live in unrelated code paths).
Need store-before-load ordering (the one thing TSO does not give you, e.g. Dekker-style mutual exclusion, or the seqlock writer)? You need a full smp_mb(). Nothing weaker orders a store followed by a load.
Dereferencing a published pointer (RCU-style)? Use rcu_dereference() (built on READ_ONCE()) and rely on the implicit address-dependency barrier — never a plain *p. See Read-Copy-Update Fundamentals.
Three-or-more-CPU agreement / IRIW-style reasoning? Use general smp_mb(); release-acquire chains do not provide multicopy atomicity.
Talking to a device via coherent DMA memory? dma_rmb()/dma_wmb(). Via MMIO registers? the readl/writel accessors or mb() — not the SMP or DMA barriers.
Pairing with an atomic RMW that has no built-in barrier (e.g. atomic_dec for a refcount)? smp_mb__before_atomic() / smp_mb__after_atomic().

The overarching rule: lock if you can, barrier if you must. The doc itself notes that taking a lock is sufficient to order shared-memory references and is far less error-prone — explicit barriers are for the lock-free fast paths (RCU readers, seqlocks, per-CPU ring buffers) where a lock’s cost is unacceptable.

Production Notes

Memory-barrier bugs are notorious precisely because they are non-deterministic and architecture-dependent. The kernel’s defense in depth is threefold. First, the API design itself: acquire/release and READ_ONCE/WRITE_ONCE make the intent explicit and let the arch layer pick the cheapest correct lowering, so most developers never write a bare smp_mb(). Second, KCSAN (the Kernel Concurrency Sanitizer, merged in 5.8) instruments accesses and barriers at runtime to flag data races and missing-barrier patterns on real workloads. Third, the Linux Kernel Memory Model ships in-tree at tools/memory-model/ with the herd7 tool and a library of litmus tests, letting developers formally check a proposed locking pattern against the model before merging — see The Linux Kernel Memory Model. The doc closes with the standing disclaimer that it “is not a specification” and that even the formal model is “the collective opinion of its maintainers rather than an infallible oracle” — a reminder that this is genuinely hard and that when in doubt, the kernel community’s advice is to ask on the list rather than guess.

The single most valuable habit: always test concurrency code on a weakly-ordered machine (ARM64) before trusting it. The x86 development box hides exactly the bugs that matter.

Three in-tree consumers worth reading

The catalogue is easier to hold in your head once you have read three real users, each of which picks a different family for a different audience.

The perf ring buffer — a four-way pairing across the kernel/user boundary. kernel/events/ring_buffer.c (v6.12) carries what is probably the clearest barrier comment in the tree, because it labels each barrier and states which one it pairs with:

 *   kernel				user
 *
 *   if (LOAD ->data_tail) {		LOAD ->data_head
 *			(A)		smp_rmb()	(C)
 *	STORE $data			LOAD $data
 *	smp_wmb()	(B)		smp_mb()	(D)
 *	STORE ->data_head		STORE ->data_tail
 *   }
 *
 * Where A pairs with D, and B pairs with C.

Reading it against this note: B pairs with C is publish-then-flag — the kernel writes sample data, smp_wmb(), then publishes data_head; userspace reads data_head, smp_rmb(), then reads the data. That is the MP pattern, and a write barrier against a read barrier is exactly right because, in the comment’s own words, “for B a WMB is sufficient since it separates two WRITEs, and for C an RMB is sufficient since it separates two READs.” A pairs with D is the mirror flow-control handshake, and it is asymmetric: A is not a barrier at all but a control dependency (“in our case (A) is a control dependency that separates the load of the ->data_tail and the stores of $data”), while D must be a full smp_mb() because “it separates the data READ from the tail WRITE” — a load followed by a store, which is the one shape a directional barrier cannot fix. Four barriers, three different strengths, each justified in one sentence: this is what a correct barrier comment looks like.

Note also the three bare barrier() calls in perf_output_put_handle(). They are not SMP barriers — they order this CPU’s own accesses against an NMI that can interrupt it mid-sequence, which is a compiler problem, not a hardware one. That is memory-barriers.txt’s first stated use for READ_ONCE/WRITE_ONCE and barrier(): “Mediating communication between process-level code and irq/NMI handlers, all running on the same CPU.”

The single-producer/single-consumer circular buffer — the documented recipe. Documentation/core-api/circular-buffers.rst (David Howells and Paul McKenney) is the kernel’s own tutorial for lock-free ring buffers, and it now uses the acquire/release forms rather than raw fences: the producer ends with smp_store_release(buffer->head, …) and the consumer begins with smp_load_acquire(buffer->head), with the tail published symmetrically. The document’s justification is precisely the one-way argument — “the smp_load_acquire() additionally forces the CPU to order against subsequent memory references” — and its scope note is the one people skip: the technique requires “just one producer and just one consumer”, with multiple producers or consumers each needing their own serialisation.

virtio — the runtime family switch. Covered above: virtio_wmb(weak_barriers) selects virt_wmb() or dma_wmb() per device, at run time, because the other end may be a hypervisor or may be real silicon. If you only read one consumer, read this one — it is the barrier taxonomy of this note compiled into a two-line if.

Resolved

An earlier revision of this note flagged as unverified the claim that x86 chooses lock; addl over MFENCE for __smp_mb() because it is cheaper. It is, and the reason is measured, not folklore. Commit 450cbdd0125c, “locking/x86: Use LOCK ADD for smp_mb() instead of MFENCE” (Michael S. Tsirkin, 27 October 2017), states: “MFENCE appears to be way slower than a locked instruction — let’s use LOCK ADD unconditionally, as we always did on old 32-bit,” with a virtio_ring benchmark going from 0.922 s to 0.579 s, “i.e. about ~60% faster” (commit 450cbdd0125c, fetched verbatim 2026-09-04). Two further details in that commit explain the exact operand: the offset is negative because “if we then read the value from SP, we get a false dependency which will slow us down”, and the negative offset is safe “since we build with the red zone disabled”. And the commit names the one semantic difference it had to work around — “LOCK ADD does not affect CLFLUSH, previous patches converted all uses of CLFLUSH to call mb()” — which is exactly why the mandatory __mb() on x86-64 remains MFENCE in v6.12 while __smp_mb() does not. The change landed in v4.15.

vault

Explorer

Memory Barriers in the Linux Kernel

Memory Barriers in the Linux Kernel

Scope: This Note Is the Catalogue

Mental Model: A Line in the Access Queue

The Complete Vocabulary

The master table

The Four-Layer Macro Dispatch

The Four Basic Barriers and Two Implicit Varieties

Acquire and Release: One-Way Permeable Barriers

How the Macros Compile: Same Source, Different Metal

x86: Strong Ordering Makes SMP Barriers Nearly Free

ARM64: Weak Ordering Needs Real Fences

The Publish-Then-Flag Pattern, Step by Step

Dependency Ordering and Why READ_ONCE Is Mandatory

Mandatory and DMA Barriers: Talking to Devices

The I/O Accessors: Where `readl()` and `writel()` Get Their Ordering

`virt_*`: Barriers for a Guest Talking to a Host

The Specialist Barriers

Implicit Barriers: What Already Has One

Failure Modes and How They Manifest

When to Choose Which Barrier

Production Notes

Three in-tree consumers worth reading

See Also

Graph View

Table of Contents

Backlinks

vault

Explorer

Memory Barriers in the Linux Kernel

Memory Barriers in the Linux Kernel

Scope: This Note Is the Catalogue

Mental Model: A Line in the Access Queue

The Complete Vocabulary

The master table

The Four-Layer Macro Dispatch

The Four Basic Barriers and Two Implicit Varieties

Acquire and Release: One-Way Permeable Barriers

How the Macros Compile: Same Source, Different Metal

x86: Strong Ordering Makes SMP Barriers Nearly Free

ARM64: Weak Ordering Needs Real Fences

The Publish-Then-Flag Pattern, Step by Step

Dependency Ordering and Why READ_ONCE Is Mandatory

Mandatory and DMA Barriers: Talking to Devices

The I/O Accessors: Where readl() and writel() Get Their Ordering

virt_*: Barriers for a Guest Talking to a Host

The Specialist Barriers

Implicit Barriers: What Already Has One

Failure Modes and How They Manifest

When to Choose Which Barrier

Production Notes

Three in-tree consumers worth reading

See Also

Graph View

Table of Contents

Backlinks

The I/O Accessors: Where `readl()` and `writel()` Get Their Ordering

`virt_*`: Barriers for a Guest Talking to a Host