Shared Memory via mmap

Shared memory is the fastest form of inter-process communication (IPC) on Linux because it eliminates the kernel from the data path entirely: two processes arrange for the same physical pages of RAM to appear in both of their virtual address spaces, so a write performed by one process is seen by the other with no system call and no copy — the bytes never leave RAM, never transit a kernel buffer, and are never serialized. The mechanism is the mmap(2) system call invoked with the MAP_SHARED flag (value 0x01, per include/uapi/linux/mman.h), which tells the kernel: “updates to this mapping are visible to other processes mapping the same region, and (in the case of file-backed mappings) are carried through to the underlying file” (mmap(2)). Each process keeps its own page-table entries (PTEs), but those PTEs are made to point at the same physical page frames, which is what makes the sharing real and the copy unnecessary. The catch that defines the whole topic: shared memory has no built-in synchronization — the kernel hands you a shared canvas and walks away, so you must layer a lock (a futex or POSIX semaphore living inside the shared region) on top of it yourself.

Scope, and where the neighbouring notes take over. This note owns the user-facing mechanism and its consequences: which flag creates a shared mapping, which backing object to choose, what a fault on a shared mapping actually does, what the kernel does and does not promise you, and how the whole thing fails in production. It deliberately does not re-derive the cache internals. The page cache’s own machinery — folios, dirty accounting, writeback thresholds, reclaim interaction — belongs to The Page Cache; the struct address_space object and its address_space_operations vtable, which is what a file-backed shared mapping ultimately hangs off, belongs to The Page Cache and address_space; the address-space and page-table plumbing of mmap itself (how a virtual address resolves to a frame, how a VMA is inserted) belongs to mmap brk and Address Space System Calls. The flag-by-flag contrast of shared against private is developed at length in MAP_SHARED vs MAP_PRIVATE, the sealing API in File Sealing with F_ADD_SEALS, and file-descriptor passing over a socket in Unix-Domain Sockets. This note links to all of them and restates only enough to stand alone.

Version pin. Every kernel-source claim below was read from Linux v6.12, a maintained long-term-support (LTS) series — releases.json on kernel.org lists 6.12.108 with moniker longterm and iseol: false as of 2026-09-04, at which point mainline is 7.3-rc1 (kernel.org releases). Anything that landed after 6.12 is called out and dated explicitly. The measured transcripts in this note were produced on a Fedora 44 machine running 7.1.8, not 6.12; that is stated where it matters, because a measurement on a newer kernel is evidence about that kernel, not about the pin.

Mental Model

The right way to picture shared memory is one tray of paint, many easels. The physical page is the paint; each process’s page table is an easel that can be positioned to look at that same tray. When the kernel sets up a MAP_SHARED mapping in two processes, it does not duplicate the page — it duplicates only the pointer to the page. Each process’s page-table entry is a separate piece of bookkeeping (process A might see the region at virtual address 0x7f00_0000, process B at 0x7f88_0000 — the virtual addresses are unrelated), but both PTEs ultimately reference the same physical frame number. A store instruction executed by A modifies the physical page; the very next load instruction executed by B reads the modified bytes, because B’s PTE resolves to that exact frame. There is no kernel involvement on the read or the write — these are ordinary CPU memory accesses through the hardware page tables.

flowchart LR
  subgraph PA["Process A address space"]
    VA["virtual addr<br/>0x7f00_0000"]
    PTEA["A's PTE"]
  end
  subgraph PB["Process B address space"]
    VB["virtual addr<br/>0x7f88_0000"]
    PTEB["B's PTE"]
  end
  PHYS["Physical page frame<br/>(one copy of the bytes in RAM)"]
  VA --> PTEA --> PHYS
  VB --> PTEB --> PHYS
  PHYS -. "file-backed only" .-> FILE["page cache entry<br/>--> backing file on disk"]

How a MAP_SHARED region maps two processes onto one physical frame. What it shows: the two processes have independent virtual addresses and independent page-table entries, but those PTEs converge on a single physical page — that convergence is the sharing. The insight to take: sharing is implemented by aliasing the page tables, not by copying memory, which is precisely why a write by A is instantly a read by B with zero kernel cost. The dashed edge applies only to file-backed mappings, where that same physical page is also a page-cache page tied to a file on disk.

The one bit that changes everything: VM_SHARED

The picture above is what MAP_SHARED buys you. Change one flag to MAP_PRIVATE and the geometry is identical until the first write, then diverges permanently. A private file mapping starts out exactly as the shared one does — both processes’ page-table entries point at the same page-cache frame, both read the same bytes — but the PTEs are installed read-only even though the mapping is PROT_WRITE, so the first store traps into the kernel’s write-protect fault handler. That handler, seeing VM_SHARED not set on the virtual memory area (VMA), allocates a brand-new anonymous frame, copies the old page’s contents into it, and re-points only the faulting process’s PTE at the copy. This is copy-on-write (COW), and it is the whole difference: after the copy, the writer is looking at private memory that no longer tracks the file or any other mapper.

flowchart TB
  subgraph BEFORE["Before any write — both flavours look the same"]
    direction LR
    A1["A: MAP_SHARED PTE<br/>(writable)"] --> F1["page-cache frame<br/>#41 — bytes: ORIGINAL"]
    B1["B: MAP_PRIVATE PTE<br/>(read-only, despite PROT_WRITE)"] --> F1
  end
  subgraph AFTER["After B stores through its private mapping"]
    direction LR
    A2["A: MAP_SHARED PTE<br/>(writable)"] --> F2["page-cache frame #41<br/>bytes: SECOND__"]
    B2["B: MAP_PRIVATE PTE<br/>(now writable)"] --> F3["fresh anonymous frame #77<br/>bytes: PRIVATE!"]
    F2 -. "no longer connected" .-x F3
  end
  BEFORE ==>|"B's first store faults:<br/>do_wp_page() sees !VM_SHARED<br/>--> allocate + copy + repoint"| AFTER

The copy-on-write divergence, drawn as pointer geometry rather than described in words. What it shows: two mappings of one object start out aliasing the same physical frame; the private mapper’s first write silently swaps its own PTE onto a private copy while the shared mapper keeps writing to the original. The insight to take: MAP_PRIVATE is not “shared but read-only” — it is “shared until you touch it, then invisibly forked.” Two consequences bite in practice: a private mapper’s changes are never seen by anyone else and never reach the file, and a private mapper stops seeing other people’s changes to any page it has already written (page-granular — untouched pages still track the file). Traced from do_wp_page() in mm/memory.c, v6.12, which branches on vma->vm_flags & (VM_SHARED | VM_MAYSHARE) before deciding whether to reuse the folio or copy it.

The divergence is easy to observe. Mapping one memfd twice — once MAP_SHARED, once MAP_PRIVATE — and writing alternately through each produces this (measured on Linux 7.1.8, x86-64; full program in the worked-examples section):

after shared write : shared=ORIGINAL   private=ORIGINAL
after private write: shared=ORIGINAL   private=PRIVATE!
after 2nd shared wr: shared=SECOND__   private=PRIVATE!    <- private no longer tracks

Line 1 is the pre-COW state: one write through the shared mapping, and the private mapping sees it, because both PTEs still point at the same frame. Line 2 is the fault: the private store did not propagate to the shared view. Line 3 is the permanent consequence: a subsequent shared write is invisible to the private mapper, because that mapper’s PTE now points at a copy. The full comparison — including the anonymous, MAP_NORESERVE and fork cases — is MAP_SHARED vs MAP_PRIVATE.

Choosing the Flag and the Backing Object

Before any mechanism, the practical question is which of a half-dozen spellings of “shared memory” to reach for. The choice is really two independent choices — which MAP_* flag and which backing object — and they are usually conflated. The flag decides whether writes are shared or copied; the object decides who can find the region, whether it survives the creating process, and whether it is on disk.

flowchart TD
  START["I want two processes to see<br/>the same bytes"] --> Q1{"Do the writes need to<br/>be visible to the peer?"}
  Q1 -->|"no — I just want a<br/>cheap read-only view"| PRIV["MAP_PRIVATE<br/>(copy-on-write; see<br/>MAP_SHARED vs MAP_PRIVATE)"]
  Q1 -->|yes| Q2{"Do the bytes need to<br/>persist on disk?"}
  Q2 -->|yes| FILE["open() a real file<br/>+ MAP_SHARED<br/>--> page-cache pages,<br/>msync/fsync for durability"]
  Q2 -->|no| Q3{"How does the peer<br/>get hold of the object?"}
  Q3 -->|"it is my own fork() child"| ANON["MAP_SHARED | MAP_ANONYMOUS<br/>before the fork<br/>--> inherited via the VMA"]
  Q3 -->|"by a well-known name"| POSIX["shm_open() on /dev/shm<br/>+ ftruncate + MAP_SHARED<br/>--> named, survives the creator"]
  Q3 -->|"I hand it an fd<br/>over a socket"| MEMFD["memfd_create()<br/>+ ftruncate + MAP_SHARED<br/>+ F_ADD_SEALS<br/>--> the modern default"]
  Q3 -->|"legacy / key_t rendezvous"| SYSV["shmget() + shmat()<br/>--> System V; avoid in new code"]
  MEMFD --> Q4{"Is the peer trusted?"}
  Q4 -->|no| SEAL["seal it:<br/>F_SEAL_SHRINK | F_SEAL_GROW<br/>| F_SEAL_WRITE"]
  Q4 -->|yes| NOSEAL["sealing optional"]
  FILE --> HUGE{"Region is hundreds of MiB<br/>and hot?"}
  ANON --> HUGE
  POSIX --> HUGE
  HUGE -->|yes| HP["add MAP_HUGETLB<br/>(+ MAP_HUGE_2MB / MAP_HUGE_1GB)"]

A decision tree for picking the flag and the backing object. What it shows: the two orthogonal decisions — sharing semantics (MAP_SHARED vs MAP_PRIVATE) and rendezvous mechanism (file, fork inheritance, /dev/shm name, passed fd, System V key) — laid out as the questions that actually distinguish them. The insight to take: the flag is almost never the hard part; the hard part is the rendezvous, and each rendezvous mechanism comes with a different lifetime and a different trust model. memfd_create wins the general case precisely because “hand the peer an fd” needs no shared namespace, leaves no stale object behind if you crash, and is the only one that can be made tamper-proof with seals.

Backing object	How the peer finds it	Lifetime	Persists to disk	Sealable	Notes
Regular file + MAP_SHARED	path	until unlinked	yes	no	Writes dirty page-cache folios; durability needs msync/fsync
MAP_SHARED | MAP_ANONYMOUS	fork() inheritance only	until last mapper exits	no	no	No name, no fd — cannot be handed to an unrelated process
shm_open() (POSIX shm)	name under /dev/shm	until shm_unlink + last close	no (tmpfs; may swap)	no	Leaks a /dev/shm entry if you crash before unlinking
memfd_create()	fd passed over a Unix socket, or /proc/PID/fd/N	until last fd/mapping closes	no (tmpfs; may swap)	yes	The modern default; see the sealing section
System V shmget()/shmat()	key_t (or IPC_PRIVATE + inheritance)	until IPC_RMID, survives all exits	no	no	Separate namespace, separate ipcs/ipcrm tooling, separate limits

The five ways to get a shared region, compared on the axes that actually decide the choice. What it shows: the rendezvous mechanism, who cleans up, and whether the object can be made immutable. The insight to take: two rows are traps. System V shared memory persists after every process that used it has exited — a crashed daemon leaves a segment that only ipcrm will reclaim, which is why sysvipc(7) exists as a separate namespace with its own tooling and why PostgreSQL’s documentation calls shared_memory_type = sysv “generally discouraged” (PostgreSQL 18 docs). POSIX shm_open has the same leak in a friendlier place: /dev/shm entries outlive the creator too, they are just visible with ls. Only memfd and anonymous shared memory are cleaned up automatically by the ordinary fd/VMA lifetime rules.

Mechanical Walk-through — Three Flavors of Shared Mapping

MAP_SHARED comes in three concrete flavors that differ only in what backs the physical pages. The sharing mechanism — aliased page tables onto common frames — is identical in all three; what changes is whether those frames are also tied to a file, and whether the data survives a reboot.

Flavor 1 — over a regular file (mmap(..., MAP_SHARED, fd, 0) where fd is a real file). Here the physical pages are page-cache pages belonging to the file. When you map a file MAP_SHARED and write through the mapping, you are dirtying page-cache pages, exactly as if you had called write(2) on the file. Two processes mapping the same file MAP_SHARED share the same page-cache pages (the file’s address_space holds exactly one cached copy of each page), so they see each other’s writes immediately — and those writes are eventually flushed to disk by the kernel’s writeback machinery, so they also persist in the file. This is the only flavor where shared memory and durable storage coincide. The The Page Cache and address_space note owns the page-cache mechanics; the IPC-relevant fact is that file-backed MAP_SHARED makes the page cache a shared-memory channel that also happens to be persistent.

Flavor 2 — over a tmpfs / shm object (RAM-backed). If you do not need persistence, you do not want disk in the path at all. POSIX shared memory gives you this: shm_open(3) creates (or opens) a named object that, on Linux, “is created in a tmpfs(5) virtual filesystem, normally mounted under /dev/shm” (shm_overview(7)). You then ftruncate(2) it to a size and mmap(..., MAP_SHARED, fd, 0) it. Because tmpfs lives in RAM (it is page-cache pages with no real disk backing — pages only spill to swap under memory pressure), there is no writeback to a block device; the “file” is the shared memory. memfd_create(2) (introduced in Linux 3.17, per memfd_create(2)) is the anonymous variant of the same idea: it returns an fd to an unnamed RAM-backed file (“the memory is anonymous memory … living in RAM”) that you mmap MAP_SHARED and pass to another process over a Unix socket — see POSIX Shared Memory and shm_open for the named flavor and the sibling memfd notes for the sealing story.

Flavor 3 — anonymous shared (MAP_SHARED | MAP_ANONYMOUS). With MAP_ANONYMOUS (value 0x20, mman-common.h) there is no file at all — “the mapping is not backed by any file; its contents are initialized to zero” (mmap(2)). On its own an anonymous mapping is just private scratch memory, but with MAP_SHARED added, the pages are shared with children created by fork(2) after the mapping is established, because the child inherits the same VMA pointing at the same frames. This is the classic way a parent and its forked workers share a region without ever touching the filesystem. Because it has no name and no fd, it can only be shared by inheritance, which is its defining limitation. The full treatment lives in the sibling Anonymous Shared Memory.

What all three share is the kernel-side setup in do_mmap(). When the MAP_SHARED (or MAP_SHARED_VALIDATE) case is taken, the kernel sets the virtual-memory-area flags VM_SHARED | VM_MAYSHARE on the new VMA (per mm/mmap.c at v6.12). VM_SHARED is the bit that tells the rest of the kernel “writes to this VMA are not copy-on-write — they go straight to the underlying page,” which is the exact inverse of the private-mapping behavior covered in MAP_SHARED vs MAP_PRIVATE. No physical page is allocated at mmap time; the mapping is lazy. The first access to a page faults, and the fault handler either finds the page already in the page cache (file/tmpfs case) or allocates a fresh zero-filled frame (anonymous case) and installs a writable PTE — writable precisely because VM_SHARED means there is nothing to copy on write.

MAP_SHARED_VALIDATE — and why the plain flag silently eats your flags

There is a third mapping type nobody reads about, and it exists because of a genuine design mistake in the original interface. MAP_SHARED is 0x01, MAP_PRIVATE is 0x02, and MAP_SHARED_VALIDATE is 0x03 — the bitwise OR of the other two, selected through the MAP_TYPE mask 0x0f (include/uapi/linux/mman.h, v6.12 and include/uapi/asm-generic/mman-common.h, v6.12). The reason for a third value is that historic mmap silently ignores flag bits it does not understand. The v6.12 code says so in its own comment:

flags_mask = LEGACY_MAP_MASK;                  /* 1 */
if (file->f_op->fop_flags & FOP_MMAP_SYNC)
        flags_mask |= MAP_SYNC;
 
switch (flags & MAP_TYPE) {
case MAP_SHARED:
        /*
         * Force use of MAP_SHARED_VALIDATE with non-legacy
         * flags. E.g. MAP_SYNC is dangerous to use with
         * MAP_SHARED as you don't know which consistency model
         * you will get. We silently ignore unsupported flags
         * with MAP_SHARED to preserve backward compatibility.
         */
        flags &= LEGACY_MAP_MASK;              /* 2 */
        fallthrough;
case MAP_SHARED_VALIDATE:
        if (flags & ~flags_mask)
                return -EOPNOTSUPP;            /* 3 */
        ...
        vm_flags |= VM_SHARED | VM_MAYSHARE;   /* 4 */

Line 1 builds the set of flags this file is allowed to accept, starting from LEGACY_MAP_MASK — the fixed list of flags that existed before validation was introduced (MAP_SHARED, MAP_PRIVATE, MAP_FIXED, MAP_ANONYMOUS, MAP_DENYWRITE, MAP_EXECUTABLE, MAP_UNINITIALIZED, MAP_GROWSDOWN, MAP_LOCKED, MAP_NORESERVE, MAP_POPULATE, MAP_NONBLOCK, MAP_STACK, MAP_HUGETLB, MAP_32BIT, MAP_ABOVE4G, MAP_HUGE_2MB, MAP_HUGE_1GB, per include/linux/mman.h, v6.12). Line 2 is the damage: with plain MAP_SHARED, every bit outside that legacy list is masked away and the call succeeds anyway. Line 3 is the fix: with MAP_SHARED_VALIDATE, an unsupported bit returns -EOPNOTSUPP instead. Line 4 is the part both paths share — the actual VM_SHARED | VM_MAYSHARE that makes the mapping shared.

flowchart TD
  CALL["mmap(..., flags, fd, off)"] --> T{"flags & MAP_TYPE<br/>(mask 0x0f)"}
  T -->|"0x01 MAP_SHARED"| MASK["flags &= LEGACY_MAP_MASK<br/><b>unknown bits silently dropped</b>"]
  T -->|"0x03 MAP_SHARED_VALIDATE"| VAL["no masking"]
  T -->|"0x02 MAP_PRIVATE"| PRIVP["private / COW path"]
  T -->|"anything else"| EINV["-EINVAL"]
  MASK --> CHK{"flags & ~flags_mask ?"}
  VAL --> CHK
  CHK -->|"yes"| EOPN["-EOPNOTSUPP<br/>(only reachable via VALIDATE)"]
  CHK -->|"no"| PERM{"PROT_WRITE requested<br/>but fd not opened writable?"}
  PERM -->|yes| EACC["-EACCES"]
  PERM -->|no| SET["vm_flags |= VM_SHARED | VM_MAYSHARE"]
  SET --> LAZY["VMA inserted; <b>no pages allocated</b><br/>population is deferred to faults"]

How do_mmap() in v6.12 disposes of the mapping-type flag. What it shows: the three legal values of the MAP_TYPE field, the silent flag-masking that only the legacy MAP_SHARED value performs, and the single line where a mapping actually becomes shared. The insight to take: if you pass a modern flag such as MAP_SYNC (persistent-memory DAX mappings, 0x080000, outside LEGACY_MAP_MASK) together with plain MAP_SHARED, the kernel drops it and hands you a successful mapping with the wrong consistency model — no error, no warning. Any new code that passes a post-2018 mapping flag should use MAP_SHARED_VALIDATE, whose entire purpose is to turn that silence into EOPNOTSUPP. For a mapping that uses only legacy flags the two are exactly equivalent, which is why plain MAP_SHARED remains correct in the overwhelming majority of code.

There is a fourth, much newer mapping type worth naming so it is not mistaken for something else: MAP_DROPPABLE (0x08), which appears in the anonymous branch of the same switch at v6.12 and marks pages the kernel may simply discard under memory pressure (it forces VM_NORESERVE | VM_WIPEONFORK | VM_DONTDUMP and refuses to combine with MAP_LOCKED or MAP_HUGETLB). It is not a sharing mechanism and is mentioned here only because it occupies a value in the same field.

What a Fault on a Shared Mapping Actually Does

mmap allocates address space, not memory. Nothing is faulted in at call time (unless you asked for MAP_POPULATE), so the first touch of every page in a shared region traps into the fault handler, and it is the handler — not mmap — that decides whether you get sharing or a copy. handle_pte_fault() in mm/memory.c, v6.12 routes a fault on a VMA with a ->fault operation into do_fault(), which makes a clean three-way decision:

} else if (!(vmf->flags & FAULT_FLAG_WRITE))
        ret = do_read_fault(vmf);        /* any read fault */
else if (!(vma->vm_flags & VM_SHARED))
        ret = do_cow_fault(vmf);         /* write to a PRIVATE mapping */
else
        ret = do_shared_fault(vmf);      /* write to a SHARED mapping */

Three lines carry the whole semantic difference between the two mapping types. A read fault is handled identically for both — populate the folio from the backing object, install a read-only PTE, and (for reads) possibly map several neighbouring pages at once via do_fault_around(). A write fault splits: do_cow_fault() preallocates a fresh anonymous folio, calls the filesystem’s ->fault to get the source page, copy_mc_user_highpage()s the bytes across, and installs a PTE pointing at the copy. do_shared_fault() does none of that — it calls ->fault to get the real folio, gives the backing address space a chance to object or prepare via ->page_mkwrite, installs a writable PTE pointing at the real page, and then calls fault_dirty_shared_page() to mark it dirty for writeback.

sequenceDiagram
  autonumber
  participant CPU as CPU (process B)
  participant FH as Fault handler<br/>handle_mm_fault / do_fault
  participant VMA as VMA flags
  participant FS as Backing object<br/>(->fault, ->page_mkwrite)
  participant PC as Physical frame

  CPU->>FH: store to 0x7f88_0000 — PTE not present
  FH->>VMA: is FAULT_FLAG_WRITE set?
  VMA-->>FH: yes
  FH->>VMA: is VM_SHARED set?
  VMA-->>FH: yes --> do_shared_fault()
  FH->>FS: __do_fault() — give me the folio for this offset
  FS-->>FH: folio (already in the page cache if A faulted it first)
  FH->>FS: ->page_mkwrite() — about to become writable
  Note over FS: filesystem may allocate blocks,<br/>start a journal transaction,<br/>or refuse (e.g. ENOSPC, F_SEAL_WRITE)
  FS-->>FH: ok
  FH->>PC: finish_fault(): install <b>writable</b> PTE --> same frame A uses
  FH->>PC: fault_dirty_shared_page(): mark folio dirty
  FH-->>CPU: retry the store — it now lands in shared RAM
  Note over CPU,PC: every subsequent access is pure hardware:<br/>no syscall, no fault, no kernel

A first write fault on a MAP_SHARED mapping, traced through do_shared_fault() at v6.12. What it shows: the sequence from trap to writable PTE, including the ->page_mkwrite callback where the backing object gets its one chance to intervene. The insight to take: the entire cost of shared memory is paid here, once per page. After finish_fault() installs the PTE there is no kernel involvement at all — which is why shared memory’s advantage is enormous for small frequent messages (no syscall per message) and modest for bulk transfer (you were going to touch that memory anyway). It also shows where a shared mapping can fail late: ->page_mkwrite runs at first-write time, not at mmap time, so a filesystem out of space or a memfd that has been sealed since you mapped it surfaces as a fault-time error — SIGBUS — rather than an mmap error code.

flowchart TD
  F["page fault on a VMA<br/>with vm_ops->fault"] --> W{"FAULT_FLAG_WRITE?"}
  W -->|no| RD["do_read_fault()<br/>fault-around + read-only PTE<br/><i>same for shared and private</i>"]
  W -->|yes| S{"vma->vm_flags<br/>& VM_SHARED?"}
  S -->|"no (MAP_PRIVATE)"| COW["do_cow_fault()<br/>alloc anon folio, copy bytes,<br/>PTE --> <b>private copy</b>"]
  S -->|"yes (MAP_SHARED)"| SH["do_shared_fault()<br/>->page_mkwrite, PTE --> <b>real folio</b>,<br/>fault_dirty_shared_page()"]
  RD --> LATER{"later store through<br/>a read-only PTE?"}
  LATER -->|"private VMA"| WP1["do_wp_page() --> wp_page_copy()<br/>or reuse if exclusively owned"]
  LATER -->|"shared VMA"| WP2["do_wp_page() --> wp_page_shared()<br/>just makes it writable + dirty"]

The complete fault taxonomy for file-backed and tmpfs-backed mappings at v6.12. What it shows: the two entry points into copy-on-write — a first-touch write fault (do_cow_fault) and a later write through an already-installed read-only PTE (do_wp_page → wp_page_copy) — and their shared-mapping counterparts, which never copy. The insight to take: VM_SHARED is consulted at every one of these four decision points, and it always means the same thing: “do not copy, write through to the object.” That single bit is the entire kernel-side implementation of shared memory. The rest is bookkeeping.

Why It Is Zero-Copy — and What “Zero-Copy” Excludes

Contrast this with a pipe or a socket. To move 1 MiB through a pipe, the sender issues write(2), which copies the bytes from the sender’s user buffer into a kernel pipe buffer; the receiver issues read(2), which copies them again from the kernel buffer into the receiver’s user buffer. That is two copies and two system calls per chunk, plus the scheduling cost of blocking and waking. With shared memory, the producer writes the 1 MiB directly into the shared region (one store pass over the data, into RAM it would have touched anyway) and the consumer reads it directly (one load pass). The kernel is never invoked for the transfer. This is why shared memory is the substrate under high-performance designs: databases (PostgreSQL’s shared buffer pool is a MAP_SHARED region every backend maps), the X server and Wayland compositors (client-rendered frames handed to the compositor as shared buffers), and inter-process ring buffers.

“Zero-copy” is precise and should not be over-read. The data is not copied between processes, but (a) for file-backed maps the data is still copied between the page cache and the disk by writeback — zero-copy is about the inter-process hop, not durability; and (b) you still pay for page faults to populate PTEs on first touch, TLB pressure, and — crucially — the cost of synchronization, discussed next.

Measured: where the win actually is

The folklore says “shared memory is the fastest IPC.” That is true, but the shape of the win is routinely misdescribed, and it is easy to measure. Two microbenchmarks on the same machine (Fedora 44, Linux 7.1.8, 32 logical CPUs, x86-64; both programs are in the worked-examples section) — one moving 512 MiB in 64 KiB chunks, one bouncing a 64-byte message back and forth 200,000 times:

Workload	Pipe	Shared memory	Ratio
Bulk: 512 MiB, 64 KiB chunks, parent → child	73 ms (6.87 GiB/s)	63 ms (7.89 GiB/s)	1.15×
Latency: 64-byte round trip, 200,000 iterations	2.26 µs/round trip	0.09 µs/round trip	25×

Measured pipe-versus-shared-memory cost on one machine. What it shows: for bulk transfer the two are nearly indistinguishable, while for small messages shared memory is more than an order of magnitude cheaper. The insight to take: the pipe’s copies are not the bottleneck at 64 KiB — a memcpy through a hot kernel buffer runs at memory bandwidth, and the shared-memory version still has to execute one full store pass over 512 MiB, so both are bandwidth-bound and land within 15% of each other. The pipe’s real cost is per operation, not per byte: four system calls per round trip, each with its entry/exit overhead, scheduler wakeups, and blocking. Shared memory’s 0.09 µs round trip contains zero system calls — just two cache-line transfers between cores. So the honest rule is: reach for shared memory when the message rate is high, not when the message size is large. If you are moving large chunks infrequently, a pipe is within noise of shared memory and vastly easier to get right. (The shared-memory figure is a busy-wait spin on an atomic sequence number; adding a blocking wait would put a futex syscall back in the path on every message that actually sleeps, which is why high-rate designs spin briefly before sleeping.)

One further caveat on the bulk number: it was taken with the region pre-faulted. On a cold MAP_SHARED | MAP_ANONYMOUS region, the first pass over 512 MiB also takes 131,072 page faults, each allocating and zeroing a 4 KiB frame, and that dominates. This is exactly what MAP_POPULATE (0x008000) and huge pages exist to amortize, and why a long-lived shared arena is set up once and reused rather than mapped per message.

Synchronization Is Your Problem

The single most important property of mmap-based shared memory is the one the kernel does not provide: there is no mutual exclusion, no ordering, no notification. If process A writes a 16-byte structure while process B reads it, B can observe a torn value — A’s first 8 bytes and B’s stale last 8 bytes — because the two stores and the two loads interleave on real hardware with real caches and real reordering. The kernel’s MAP_SHARED contract is only “you see the same bytes”; it says nothing about when or atomically. Two tools fix this, and both live inside the shared region itself:

• A futex — a 32-bit word in the shared memory that userspace atomically compare-and-swaps for the uncontended fast path, falling into the futex(2) syscall only to sleep or wake a waiter on contention. This is how a cross-process pthread_mutex_t (created with PTHREAD_PROCESS_SHARED) works: the lock word lives in the shared region and is a futex. See Futex and OS Synchronization Primitives.
• A POSIX semaphore placed in the shared region (sem_init with pshared=1), which is itself futex-backed on Linux.

The full treatment of how to build correct cross-process locks over a shared region is the sibling Synchronizing Shared Memory with Futexes and Semaphores. The takeaway here: a shared mapping without a lock is a data race, full stop.

sequenceDiagram
  autonumber
  participant A as Process A (writer)
  participant M as Shared frame<br/>{ ready, len, buf[] }
  participant B as Process B (reader)

  Note over A,B: the bug — no ordering, no lock
  A->>M: buf[] = "hello world"  (may be reordered by CPU/compiler)
  A->>M: len = 11
  A->>M: ready = 1
  B->>M: read ready == 1
  B->>M: read len, buf[]
  Note over B: B may observe ready=1 while<br/>buf[] is still the OLD contents:<br/>the stores can become visible out of order

  Note over A,B: the fix — release/acquire around the flag
  A->>M: buf[] = "hello world" then len = 11
  A->>M: atomic_store_explicit(&ready, 1, <b>memory_order_release</b>)
  B->>M: atomic_load_explicit(&ready, <b>memory_order_acquire</b>) == 1
  B->>M: read len, buf[] — now guaranteed to see A's stores

Why “the kernel guarantees you see the same bytes” is not the same as “you see them in the right order.” What it shows: the classic publish-a-buffer bug, where the flag that announces the data becomes visible before the data itself, and the release/acquire pairing that fixes it. The insight to take: MAP_SHARED gives coherence (both processes address the same cache-coherent physical memory) but not ordering. Coherence is a hardware property of the memory system; ordering is a property of your program, and you get it from acquire/release atomics, explicit fences, or a lock — never from the mapping flag. The rules are identical to multithreaded shared memory within one process, because the hardware cannot tell the difference; see Acquire Release and Fence Semantics and Memory Barriers in the Linux Kernel. Note also that partial-word tearing is a separate hazard: a plain non-atomic store of a 16-byte struct is not one operation, and a reader can catch it half-written.

Two further wrinkles specific to cross-process synchronization, as opposed to cross-thread. First, a pthread_mutex_t must be initialized with PTHREAD_PROCESS_SHARED for its futex word to be interpreted correctly by both processes — the default is PTHREAD_PROCESS_PRIVATE, and glibc uses the FUTEX_PRIVATE_FLAG fast path for private futexes, which keys the wait queue on the process’s own mm and therefore will never wake a waiter in another process. Futexes Across Process Boundaries develops this. Second, a process holding a lock in shared memory can die while holding it, leaving the region permanently wedged; POSIX robust mutexes (pthread_mutexattr_setrobust) exist precisely for this, and the kernel-side machinery is the robust-futex list walked at task exit. A design that puts locks in shared memory without a robustness story has an availability bug waiting to happen.

memfd_create and File Sealing — the Modern Answer

Of the five backing objects in the table above, one has become the default in every modern userspace stack that shares memory between programs that do not fully trust each other, and it is worth understanding why rather than just that. The problem memfd_create(2) and file sealing solve is not performance — it is that plain shared memory has no way to make a promise.

The threat model, stated precisely

Suppose a client renders a window into a shared buffer and hands it to a compositor. The compositor must read the buffer. With ordinary POSIX shared memory the client retains write access to the same pages, and two concrete attacks follow, both spelled out in memfd_create(2):

• Time-of-check-to-time-of-use (TOCTOU). “An untrusted peer might modify the contents of the shared memory at any time,” so any value the compositor validates (a length, an offset, a pixel format) can be changed between the check and the use. The historical defence is to copy the whole buffer out before validating it — which throws away the entire point of shared memory.
• SIGBUS by truncation. “The latter possibility leaves the local process vulnerable to SIGBUS signals when an attempt is made to access a now-nonexistent location in the shared memory region.” A client that ftruncates the object smaller after the compositor has mapped it can crash the compositor on the next access, and “dealing with this possibility necessitates the use of a handler for the SIGBUS signal” — a genuinely awful thing to have to write.

Sealing removes both. The kernel comment in mm/memfd.c, v6.12 states the design goal directly: “Sealing allows multiple parties to share a tmpfs or hugetlbfs file but restrict access to a specific subset of file operations. Seals can only be added, but never removed. This way, mutually untrusted parties can share common memory regions with a well-defined policy. A malicious peer can thus never perform unwanted operations on a shared object.”

The seals, and exactly what each forbids

Seals live on the inode, so every open file descriptor and every mapping of that object sees the same set, and the set is monotonic — additions only. F_ADD_SEALS and F_GET_SEALS are fcntl(2) operations; note that in current man-pages they have moved out of the fcntl(2) page into their own F_GET_SEALS(2const) page (verified 2026-09-04 against the man-pages git tree; older material will point you at fcntl(2), where the text no longer is).

Seal	Value	Forbids	Notes
F_SEAL_SEAL	0x0001	adding any further seal	Present by default unless MFD_ALLOW_SEALING was passed
F_SEAL_SHRINK	0x0002	ftruncate/O_TRUNC shrinking the object	This is the anti-SIGBUS seal
F_SEAL_GROW	0x0004	write past EOF, truncate, fallocate growing it	Pairs with SHRINK to pin the size
F_SEAL_WRITE	0x0008	write, hole-punching fallocate, and new shared writable mmap	Fails EBUSY if a writable shared mapping already exists
F_SEAL_FUTURE_WRITE	0x0010	the same, except through mappings created before the seal	Since Linux 5.1; lets the producer keep writing
F_SEAL_EXEC	0x0020	changing the file’s execute mode bits	Since Linux 6.3; implies GROW|SHRINK|WRITE|FUTURE_WRITE

The complete seal set at v6.12, values read from include/uapi/linux/fcntl.h and semantics from the man page. What it shows: each seal names one class of operation and turns it into EPERM forever. The insight to take: the useful seals come in a set, not individually. F_SEAL_WRITE alone is not enough to freeze content — the comment in the man-pages source spells out the hole: with only WRITE sealed you could still ftruncate 100 bytes off and then grow the file back by 100 bytes, replacing those bytes with zeroes. The canonical “this buffer is now immutable” incantation is therefore F_SEAL_SHRINK | F_SEAL_GROW | F_SEAL_WRITE together, which is exactly the 0xe a real program observes below.

The F_SEAL_EXEC implication is enforced in code, not just documented — memfd_add_seals() at v6.12 contains if (seals & F_SEAL_EXEC && inode->i_mode & 0111) seals |= F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE|F_SEAL_FUTURE_WRITE; with the comment “SEAL_EXEC implys SEAL_WRITE, making W^X from the start.” Relatedly, memfd_create gained MFD_NOEXEC_SEAL (0x0008) and MFD_EXEC (0x0010) and a vm.memfd_noexec sysctl that can force MFD_NOEXEC_SEAL on every memfd in a PID namespace, or reject any call without it with -EACCES — a hardening measure against using memfd as an anonymous executable staging area.

Measured: sealing enforced, step by step

Running the seal sequence on a live kernel (Linux 7.1.8, x86-64) makes the contract concrete:

== file sealing ==
  F_SEAL_WRITE while shared writable mapping exists -> -1 (Device or resource busy)
  after munmap of the writable mapping             -> 0 (ok)
  F_GET_SEALS -> 0xe
  ftruncate grow after F_SEAL_GROW                 -> -1 (Operation not permitted)
  new MAP_SHARED PROT_WRITE after F_SEAL_WRITE     -> Operation not permitted
  add F_SEAL_SEAL -> 0; then add F_SEAL_EXEC -> -1 (Operation not permitted)

Every line is a rule you can name. The EBUSY on line 1 is not a bug — memfd_add_seals() calls mapping_deny_writable() and then memfd_wait_for_pins(), which tags every folio whose reference count exceeds its map count and waits up to five escalating scans (LAST_SCAN = 4, “about 150ms max” per the source comment) for those references to drop, returning -EBUSY if any survive. The kernel comment explains what it is really guarding against: “via get_user_pages(), drivers might have some pending I/O without any active user-space mappings (eg., direct-IO, AIO).” In other words, sealing a buffer for writing must not race with a DMA that is still in flight into it. Line 3’s 0xe is SHRINK|GROW|WRITE — the immutability set. Line 6 shows the monotonicity: once F_SEAL_SEAL is on, even a further seal is refused with EPERM.

F_SEAL_FUTURE_WRITE (Linux 5.1) is the seal that makes the producer/consumer case ergonomic, and it is the one worth knowing about: it blocks new writable mappings and write(2) while leaving existing shared writable mappings functional, so “one process can create a memory buffer that it can continue to modify while sharing that buffer on a ‘read-only’ basis with other processes.” That is the shape of a video frame pipeline, and it is why you do not have to unmap and remap on every frame.

The full handoff

sequenceDiagram
  autonumber
  participant P as Producer (untrusted by C)
  participant K as Kernel (tmpfs inode)
  participant S as Unix-domain socket
  participant C as Consumer (e.g. compositor)

  P->>K: memfd_create("frame", MFD_CLOEXEC|MFD_ALLOW_SEALING)
  Note over K: anonymous tmpfs inode,<br/>seals = 0 (empty, because ALLOW_SEALING)
  P->>K: ftruncate(fd, size)
  P->>K: mmap(MAP_SHARED, PROT_READ|PROT_WRITE)
  P->>K: fill the buffer
  P->>K: fcntl(F_ADD_SEALS,<br/>SHRINK|GROW|FUTURE_WRITE)
  Note over K: memfd_wait_for_pins():<br/>-EBUSY if a DMA still pins a folio
  P->>S: sendmsg() with SCM_RIGHTS carrying the fd
  S->>C: recvmsg() --> a NEW fd, same inode
  C->>K: fcntl(F_GET_SEALS) --> verify the promise
  Note over C: only now is it safe to map<br/>without a SIGBUS handler
  C->>K: mmap(MAP_SHARED, PROT_READ)
  C->>C: parse in place — no defensive copy needed

The memfd + seals + SCM_RIGHTS handoff that Wayland, D-Bus and Chromium all converge on. What it shows: the ordering that makes it safe — create, size, fill, seal, then pass; and on the receiving side, verify the seals before mapping. The insight to take: the fd-passing step is what makes memfd composable — there is no shared namespace to collide in, no /dev/shm entry to leak, and the object dies with the last reference. But the security property comes from step 6 combined with step 9: a consumer that maps without calling F_GET_SEALS has gained nothing, because a sender is free to pass an unsealed fd. Sealing is a verifiable promise, not an automatic one. Full mechanics of the fd transfer are in Unix-Domain Sockets and Passing memfd Buffers Between Processes; the seal API in depth is File Sealing with F_ADD_SEALS and memfd_create and Anonymous Memory Files.

Two implementation details that trip people up. First, memfd_create returns a file whose initial size is zero; you must ftruncate before mapping anything useful, and a zero-length object can still be mmapped successfully — the failure arrives later as SIGBUS. Second, the initial seal set depends on the flags: without MFD_ALLOW_SEALING every fresh tmpfs inode starts life with info->seals = F_SEAL_SEAL (set unconditionally in __shmem_get_inode() in mm/shmem.c, v6.12), i.e. sealing is permanently disabled — memfd_create clears that bit only when MFD_ALLOW_SEALING or MFD_NOEXEC_SEAL was requested; MFD_ALLOW_SEALING clears that bit so the set starts empty. You cannot decide to start sealing later.

Huge Pages for Shared Regions

A large shared region — a database buffer pool, a model-weights arena, a big ring buffer — pays a page-table and translation-lookaside-buffer (TLB) tax that scales with its size in 4 KiB pages. A 64 GiB region is 16,777,216 PTEs; at 2 MiB pages it is 32,768. MAP_HUGETLB (0x040000, mman-common.h, v6.12) asks for the mapping to be backed by pages from the hugetlb pool instead, and it composes with MAP_SHARED exactly as you would expect.

The size selector is an unusual piece of ABI worth reading carefully, because it is not a flag but a number packed into the flags word. MAP_HUGE_SHIFT and MAP_HUGE_MASK (from asm-generic/hugetlb_encode.h, re-exported by include/uapi/linux/mman.h, v6.12) carve out bits 26–31 of the flags argument, and the value stored there is the base-2 logarithm of the page size. So MAP_HUGE_2MB is 21 << 26 and MAP_HUGE_1GB is 30 << 26. The header enumerates every encoding the kernel knows — 16 KiB, 64 KiB, 512 KiB, 1 MiB, 2 MiB, 8 MiB, 16 MiB, 32 MiB, 256 MiB, 512 MiB, 1 GiB, 2 GiB, 16 GiB — but the header’s own comment is emphatic that presence in the list is not availability: “It is the responsibility of the application to know which sizes are supported on the running system.” Omitting the size bits entirely gives you the system default huge page size.

flowchart LR
  subgraph SMALL["MAP_SHARED, 4 KiB pages"]
    direction TB
    V1["1 GiB region"] --> P1["262,144 PTEs<br/>~2 MiB of page tables<br/>per mapping process"]
  end
  subgraph BIG["MAP_SHARED | MAP_HUGETLB | MAP_HUGE_2MB"]
    direction TB
    V2["1 GiB region"] --> P2["512 PTEs<br/>~4 KiB of page tables<br/>per mapping process"]
  end
  SMALL -->|"same bytes, 512x fewer<br/>translations to cache"| BIG
  BIG --> C1["pool must be pre-reserved:<br/>vm.nr_hugepages or hugetlbfs mount"]
  BIG --> C2["not swappable, not reclaimable"]
  BIG --> C3["mmap fails ENOMEM if the pool<br/>is exhausted — no graceful fallback"]

What MAP_HUGETLB buys and what it costs, for a 1 GiB shared region. What it shows: the page-table arithmetic that motivates huge pages, multiplied by the number of processes mapping the region, alongside the three operational constraints. The insight to take: the page-table saving is per-mapping-process, so it compounds exactly in the case shared memory is used for — a hundred backends mapping one pool. But MAP_HUGETLB memory comes from a pre-reserved, non-reclaimable pool, so it converts a soft failure (slow, swapping) into a hard one (mmap returns ENOMEM), and that reservation has to be arranged out of band. See hugetlbfs and Reserved Huge Pages for pool management and Transparent Huge Pages for the automatic alternative, which needs no flag but gives no guarantee.

memfd_create has the matching MFD_HUGETLB (0x0004) and the same MFD_HUGE_* encoding, so a sealed, huge-page-backed, fd-passed shared buffer is expressible in one call. Note that hugetlbfs inodes carry seals too — memfd_file_seals_ptr() at v6.12 returns &HUGETLBFS_I(file_inode(file))->seals for huge-page files, so sealing works on both tmpfs and hugetlbfs objects and nothing else; on any other filesystem the fcntl returns EINVAL.

PostgreSQL is a good calibration point for how this is used in practice: its main shared memory region defaults to shared_memory_type = mmap (“anonymous shared memory allocated using mmap”), and its huge_pages setting — default try — “is only supported when shared_memory_type is set to mmap” on Linux, with try meaning request huge pages and fall back silently, on meaning refuse to start without them (PostgreSQL 18 documentation). That try/on split is the general shape of the trade: try keeps the soft failure, on makes the reservation a startup precondition.

Flushing to a Backing File — msync

For file-backed MAP_SHARED mappings there is a durability subtlety. When you write through the mapping you dirty page-cache pages, but those dirty pages are written back to disk asynchronously by the kernel’s flusher threads on their own schedule — and the man page is explicit that “without use of [msync], there is no guarantee that changes are written back before munmap(2) is called” (msync(2)). To force the issue you call msync(addr, length, flags):

• MS_SYNC — “requests an update and waits for it to complete” (msync(2)). This is the durable, blocking flush: when it returns, your data is on the backing store (subject to the device honoring its own cache flush). Use this when you need the persistence guarantee, e.g. a memory-mapped database log.
• MS_ASYNC — “specifies that an update be scheduled, but the call returns immediately.” It nudges writeback to start but does not wait. On modern Linux this is nearly a no-op because dirty pages are already on the writeback path; it does not give you a durability guarantee.
• MS_INVALIDATE — “asks to invalidate other mappings of the same file (so that they can be updated with the fresh values just written).” Rarely needed because coherent mappings already share the page cache.

The flags rule is exact: you must pass exactly one of MS_SYNC or MS_ASYNC, optionally OR-ed with MS_INVALIDATE; passing both MS_SYNC and MS_ASYNC is EINVAL (msync(2)). The numeric values are MS_ASYNC = 1, MS_INVALIDATE = 2, MS_SYNC = 4 (mman-common.h, v6.12) — note that they are not a clean bit-per-mode encoding, which is why MS_SYNC | MS_ASYNC is a detectable error rather than a plausible combination. For the RAM-backed flavors (tmpfs/memfd/anonymous) msync is meaningless for durability — there is no disk to flush to — though it is still legal to call.

flowchart TD
  Q0["I wrote through a<br/>MAP_SHARED mapping"] --> Q1{"What is backing it?"}
  Q1 -->|"MAP_ANONYMOUS,<br/>memfd, /dev/shm"| NONE["<b>msync is pointless.</b><br/>There is no disk.<br/>Peers already see the bytes."]
  Q1 -->|"a real file"| Q2{"Do I need the bytes to<br/>survive a power cut?"}
  Q2 -->|"no — I only need my<br/>peer to see them"| NONE2["<b>msync is still pointless.</b><br/>Coherence is immediate:<br/>same page-cache folio."]
  Q2 -->|yes| Q3{"Can I tolerate<br/>blocking here?"}
  Q3 -->|yes| SYNC["msync(addr, len, <b>MS_SYNC</b>)<br/>returns when writeback completed"]
  Q3 -->|"no — commit point<br/>is elsewhere"| ASYNC["msync(..., MS_ASYNC) starts it,<br/><b>guarantees nothing</b>;<br/>you still need a real barrier later"]
  SYNC --> DEV{"Device has a<br/>volatile write cache?"}
  DEV -->|yes| FSYNC["msync alone may not be enough —<br/>see fsync fdatasync and Durability"]
  DEV -->|no| DONE["durable"]

When msync is and is not needed. What it shows: the two questions that eliminate msync from most designs — is there a disk at all, and do you actually need durability rather than visibility. The insight to take: the single most common misuse of msync is calling it to make a peer see your writes. It has nothing to do with that. Peers sharing the same object see stores the instant the store instruction retires, because they are addressing the same physical frame; msync only pushes dirty page-cache folios toward the block device. If your peer is not seeing your data, the problem is ordering or the wrong object, not a missing flush. The mirror-image mistake is trusting msync(MS_ASYNC) as a durability barrier — it schedules writeback and returns, which on a modern kernel is close to a no-op because those folios were already on the writeback path.

Worked Example — Two Processes Sharing a Counter

The canonical minimal example: a parent and child share a 64-bit counter via MAP_SHARED | MAP_ANONYMOUS so increments are visible across the fork. This is flavor 3, with deliberately no synchronization to expose the race.

#include <sys/mman.h>      /* mmap, MAP_SHARED, MAP_ANONYMOUS, PROT_* */
#include <unistd.h>        /* fork, _exit */
#include <stdint.h>
#include <stdio.h>
 
int main(void) {
    /* PROT_READ|PROT_WRITE: pages may be read and written (0x1|0x2).
       MAP_SHARED|MAP_ANONYMOUS: shared, file-less, zero-initialized.
       fd = -1, offset = 0: required form for anonymous maps.        */
    uint64_t *counter = mmap(NULL, sizeof(*counter),
                             PROT_READ | PROT_WRITE,
                             MAP_SHARED | MAP_ANONYMOUS, -1, 0);
    if (counter == MAP_FAILED) { perror("mmap"); return 1; }
    *counter = 0;                         /* parent initializes (pre-fork) */
 
    pid_t pid = fork();                   /* child inherits the SAME VMA  */
    if (pid == 0) {                       /* --- child --- */
        for (int i = 0; i < 1000000; i++)
            (*counter)++;                 /* RACE: read-modify-write, unsynced */
        _exit(0);
    }
    /* --- parent --- */
    for (int i = 0; i < 1000000; i++)
        (*counter)++;                     /* RACE: same word, no lock */
 
    /* (waitpid omitted for brevity) */
    sleep(1);
    printf("counter = %lu (expected 2000000)\n", *counter);
    munmap(counter, sizeof(*counter));    /* tear down the mapping */
    return 0;
}

Line-by-line, the important parts: the mmap establishes one anonymous shared frame before fork, so both processes inherit PTEs onto it. *counter = 0 and every (*counter)++ are ordinary memory accesses — the kernel is never called, which is the whole point. The printed counter will almost always be less than 2,000,000: (*counter)++ compiles to load-increment-store, and the parent’s and child’s read-modify-write cycles interleave on the shared word, losing increments. This is not a bug in shared memory — it is the absence of a lock, demonstrating exactly why Synchronizing Shared Memory with Futexes and Semaphores exists. Wrap the increment in a PTHREAD_PROCESS_SHARED mutex placed in a second shared field, and the count becomes exact.

A file-backed variant differs only in setup: int fd = open("data.bin", O_RDWR); ftruncate(fd, 4096); void *p = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);. Now writes to p dirty the page cache for data.bin, are visible to any other process mapping the same file, and become durable on disk after an msync(p, 4096, MS_SYNC) (or eventual writeback).

Worked Example 2 — a sealed memfd, end to end

This is the program that produced the sealing transcript above. It is short because the API is short; the value is in which call fails and why.

#define _GNU_SOURCE
#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <linux/memfd.h>
 
/* 1. Create an anonymous RAM-backed file. MFD_ALLOW_SEALING is REQUIRED here:
      without it the inode starts with F_SEAL_SEAL and can never be sealed.  */
int fd = syscall(SYS_memfd_create, "frame", MFD_CLOEXEC | MFD_ALLOW_SEALING);
 
/* 2. It starts at length 0. Mapping a zero-length file SUCCEEDS and then
      SIGBUSes on first touch, so size it first.                            */
ftruncate(fd, 4096);
 
/* 3. Map it shared+writable and fill it. This mapping is what will block
      F_SEAL_WRITE later, because a writable shared mapping pins the folios. */
char *buf = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
memcpy(buf, payload, payload_len);
 
/* 4. Drop the writable mapping BEFORE sealing writes. Otherwise
      memfd_add_seals() -> mapping_deny_writable() fails and you get EBUSY.
      (Or skip this and use F_SEAL_FUTURE_WRITE, which tolerates it.)        */
munmap(buf, 4096);
 
/* 5. The immutability set: no shrink (no SIGBUS for the peer), no grow,
      no writes. Seals are per-inode and can never be removed.               */
fcntl(fd, F_ADD_SEALS, F_SEAL_SHRINK | F_SEAL_GROW | F_SEAL_WRITE);
 
/* 6. Hand `fd` to the peer over a Unix socket with SCM_RIGHTS.
      The peer MUST call fcntl(fd, F_GET_SEALS) and check the bits before
      trusting the buffer — an unsealed fd looks identical.                  */

Step 4 is the one that surprises everyone the first time. F_SEAL_WRITE is not merely a future-tense promise; the kernel enforces that no writable path currently exists, which means unmapping your own writable mapping and waiting out any in-flight get_user_pages() references. If your producer needs to keep writing — a compositor client updating a frame in place — F_SEAL_FUTURE_WRITE is the seal designed for exactly that case and skips step 4 entirely.

Failure Modes and Common Misunderstandings

flowchart TB
  S{"Symptom"}
  S -->|"peer sees stale<br/>or no data"| M1["Wrong object, wrong offset,<br/>one side used MAP_PRIVATE,<br/>or missing release/acquire.<br/><b>Never a missing msync.</b>"]
  S -->|"SIGBUS on a<br/>valid-looking address"| M2["Address is past the backing<br/>object's current size.<br/>Forgot ftruncate, or the peer<br/>shrank it under you."]
  S -->|"counter/struct has<br/>impossible values"| M3["Unsynchronised read-modify-write<br/>or a torn multi-word store.<br/>Needs a lock, not a barrier alone."]
  S -->|"region wedged;<br/>everyone blocked"| M4["A process died holding a<br/>lock inside the region.<br/>Needs robust mutexes."]
  S -->|"ENOSPC / OOM<br/>from shm_open"| M5["/dev/shm tmpfs limit<br/>(default half of RAM).<br/>Counts against memory, not disk."]
  S -->|"F_ADD_SEALS<br/>returns EBUSY"| M6["A writable shared mapping or a<br/>pinned folio still exists.<br/>munmap, or use FUTURE_WRITE."]
  S -->|"data lost after<br/>power cut"| M7["File-backed writeback is async.<br/>msync(MS_SYNC) at the commit<br/>point, or accept the loss."]
  S -->|"segment survives<br/>process exit"| M8["System V shmget or a leaked<br/>/dev/shm entry.<br/>ipcrm / shm_unlink."]

A symptom-to-cause map for shared-memory bugs. What it shows: the eight presentations that recur and the mechanism behind each, all developed in prose below. The insight to take: notice how few of these are kernel problems. Six of the eight are contract violations by the application — wrong object, wrong size, wrong synchronisation, wrong cleanup — which is the price of an interface whose whole design is “the kernel gets out of the way.” The two that are genuinely about kernel behaviour (SIGBUS past end-of-object and asynchronous writeback) are both cases where the kernel is doing exactly what it documents and the application assumed otherwise.

“I mapped it MAP_SHARED but the other process doesn’t see my writes.” Almost always one of: the two processes are not actually mapping the same object (different files, or one used MAP_PRIVATE — see MAP_SHARED vs MAP_PRIVATE); the writer wrote before the reader mapped and the reader mapped a different file offset; or there is no synchronization and the reader simply hasn’t been told the data is ready. MAP_SHARED coherence is immediate for the same backing object — if you see staleness, you are looking at the wrong object or a missing memory barrier.

SIGBUS on access. If you mmap a file and then ftruncate it smaller (or never sized it with ftruncate after shm_open, which starts at zero length), touching a mapped address beyond the file’s current size delivers SIGBUS, not SIGSEGV. The classic bug: shm_open a fresh object, forget the ftruncate, mmap 4096 bytes successfully (mapping a zero-length file is allowed), then crash on first access. Size the object before mapping.

The distinction matters diagnostically, and it is worth internalising why the kernel picks a different signal. SIGSEGV means “this virtual address is not part of any mapping” — an address-space error, caught by the VMA lookup. SIGBUS means “the address is inside a perfectly valid mapping, but there is no backing object at that offset to fault a page in from” — a backing-store error, produced deep in do_fault() when the filesystem’s ->fault handler has nothing to return. So SIGBUS on an mmaped region is nearly always a size bug, never a pointer bug, and no amount of staring at the pointer arithmetic will find it.

Both variants are trivially reproducible (measured on Linux 7.1.8, using a memfd sized to exactly one page and mapped for two):

== SIGBUS past the end of the backing object ==
  touch offset 0    (inside  4096-byte object): ok
  touch offset 4096 (beyond  4096-byte object): SIGBUS
  touch offset 0 after ftruncate(fd,0): SIGBUS  <- the truncation race

The first two lines are the “forgot to size it” bug in miniature: mmap happily returned an 8 KiB mapping over a 4 KiB object, and the second page is a landmine. The third line is the far nastier one, and is the entire reason F_SEAL_SHRINK exists: an address that worked a microsecond ago stops working because another process shrank the object. There is no defensive coding that fixes this — you cannot check the size and then dereference, because the peer can shrink in between. Your options are exactly three: install a SIGBUS handler and structure your accesses so you can recover from a longjmp out of one; trust the peer; or make the peer’s shrink impossible with F_SEAL_SHRINK. The third is the only one that composes, which is why sealing became the standard answer.

Writes silently lost on crash (file-backed). Because writeback is asynchronous, a power loss between your last store and the next writeback loses data. If durability matters, msync(..., MS_SYNC) at the commit point; do not assume mapped writes are on disk.

offset not page-aligned. mmap’s offset “must be a multiple of the page size” (mmap(2)); a non-aligned offset is EINVAL. To map a non-page-aligned region of a file, map from the aligned offset below it and index into the mapping.

tmpfs full. /dev/shm is a tmpfs with a size limit (often half of RAM by default). A large shm_open+ftruncate+write can hit ENOSPC or, worse, trigger the OOM behavior of tmpfs. Size /dev/shm deliberately for memory-heavy shared-memory designs. The subtlety that makes this bite is that a tmpfs “file” is not disk-like at all: it is page-cache folios with no backing device, so its bytes are charged against memory, are visible in Shmem in /proc/meminfo, and can only be reclaimed by pushing them to swap. A container with a small memory limit and a large /dev/shm write will be OOM-killed for what looks, in df, like ordinary free space. tmpfs In-Memory Filesystem covers the filesystem itself and its mount options.

F_ADD_SEALS returns EBUSY. Not a transient error — it means a writable reference to the object still exists, either your own MAP_SHARED | PROT_WRITE mapping or a folio still pinned by an in-flight get_user_pages() (direct I/O, AIO, a driver DMA). memfd_add_seals() will wait through five escalating scans before giving up, so a retry loop is usually the wrong fix; find the writable mapping and unmap it, or switch to F_SEAL_FUTURE_WRITE.

The region is wedged and every process is blocked. A lock living inside the shared region is not cleaned up when its holder dies — the kernel has no idea that a particular 32-bit word was a mutex. Unless the mutex was created with pthread_mutexattr_setrobust(..., PTHREAD_MUTEX_ROBUST), so that the next acquirer gets EOWNERDEAD and can call pthread_mutex_consistent(), the region stays locked forever. This is the single most common way a shared-memory design turns a crash of one process into an outage of all of them.

The segment outlives everything. System V shared memory created with shmget() persists until an explicit shmctl(..., IPC_RMID, ...) — process exit does not free it, and neither does a reboot of the daemon that made it. ipcs -m lists the orphans and ipcrm removes them. POSIX shm_open has a gentler version of the same problem: the /dev/shm entry survives until shm_unlink, so a crash between shm_open and shm_unlink leaves a named object holding RAM. memfd and MAP_SHARED | MAP_ANONYMOUS are the only flavours with no leak mode, because their lifetime is exactly the lifetime of the fds and mappings referring to them.

Alternatives and When to Choose Them

Shared memory wins when the data volume is large or the latency must be minimal and the processes are co-located on one host. It loses when you need a stream with backpressure (use a pipe or Unix socket — they carry flow control for free), when you need to communicate across machines (shared memory is host-local; use sockets), or when you want the kernel to handle framing and you do not want to write your own synchronization (a socket is far less error-prone for small, infrequent messages). The honest trade: shared memory is the fastest channel and the easiest to get subtly wrong, because you are responsible for layout, lifetime, and locking. Reach for it when profiling shows the copy or the syscall is the bottleneck, not by default.

A close cousin is the splice family, which achieves zero-copy movement between file descriptors inside the kernel without mapping anything into userspace — a better fit when you are shoveling bytes A-to-B (file-to-socket) rather than maintaining a shared data structure.

Mechanism	Copies per message	Framing	Backpressure	Notification	Cross-host	You must write
MAP_SHARED region	0	none — your layout	none	none	no	layout, locking, lifetime, versioning
Pipe	2	byte stream	yes (blocks when full)	yes (readable fd)	no	message framing
Unix-domain socket	2	SOCK_SEQPACKET gives it free	yes	yes (pollable fd)	no	little
memfd + SCM_RIGHTS over a socket	0 for the payload, 2 for the tiny control message	socket frames the control message	yes, on the control channel	yes	no	buffer layout + seals
TCP socket	2 (plus network)	stream	yes	yes	yes	framing, serialization

The IPC mechanisms compared on the axes that decide between them. What it shows: what each one gives you for free and what it leaves as your problem. The insight to take: the fourth row is the design almost every modern desktop and browser stack converged on, and it is not a compromise — it is the best of both. The payload moves with zero copies through shared memory; the notification and backpressure ride on a socket, which is exactly what sockets are good at and exactly what raw shared memory cannot do. If you find yourself building a condition variable and a ring buffer inside a shared region so that one process can tell another “there is data now,” stop and consider whether an fd-passing design would let the kernel do that part for you. The pure shared-memory design earns its keep only when the message rate is high enough that a syscall per notification is itself the cost you are trying to remove — see the measured 2.26 µs versus 0.09 µs above.

Production Notes

PostgreSQL allocates its main shared buffer pool as one large shared-memory region every backend process maps, coordinating access with its own lightweight locks (LWLocks) built over atomics and futex-like waits in the shared region — a textbook example of “shared memory plus your own synchronization.” Its configuration surface is a useful window into the trade-offs: shared_memory_type selects between mmap (“anonymous shared memory allocated using mmap”), sysv (shmget) and windows, with “the first supported option [as] the default for that platform” — on Linux that is mmap — and the documentation is blunt that sysv “is generally discouraged because it typically requires non-default kernel settings to allow for large allocations.” A second, separate setting dynamic_shared_memory_type covers regions created after startup (for parallel query), where the options are posix (shm_open), sysv, windows and mmap-a-real-file, and the file-backed option is discouraged “because the operating system may write modified pages back to disk repeatedly, increasing system I/O load” (PostgreSQL 18 documentation). That last remark is the file-backed flavour’s central production hazard stated by people who hit it: a MAP_SHARED mapping of a real file is also a dirty-page generator, and a hot region backed by a file will be written to disk over and over by writeback for no benefit at all.

Graphics stacks are the other canonical consumer. Wayland’s core protocol defines wl_shm_pool as an object that “encapsulates a piece of memory shared between the compositor and client,” created by wl_shm.create_pool(id, fd, size) where “the server will mmap size bytes of the passed file descriptor, to use as backing memory for the pool” (wayland.xml, protocol source). Two design decisions in that protocol are direct consequences of everything above. First, wl_shm_pool.resize “can only be used to make the pool bigger” — the protocol forbids shrinking, because a shrink would SIGBUS the compositor, which is the truncation race made into a wire-protocol rule. Second, wl_shm defines an invalid_fd error meaning “mmapping the file descriptor failed,” acknowledging that the compositor must survive being handed a bad fd by an untrusted client. A compositor that additionally checks F_GET_SEALS for F_SEAL_SHRINK can drop its SIGBUS handler entirely, which is precisely the migration the ecosystem has been making.

Chromium reaches the same conclusion from the sandbox side. Its PlatformSharedMemoryRegion documents a mode that “creates a new kUnsafe region backed by an anonymous file (memfd)” and notes it “requires that the seccomp policy allows memfd_create(2) (the baseline policy does)” and returns “an invalid region if the kernel does not support memfd_create()” (base/memory/platform_shared_memory_region.h). The interesting part is the type system built on top: regions come in kReadOnly, kWritable and kUnsafe modes, a writable region “can be demoted to ReadOnlySharedMemoryRegion without violating security policies” via ConvertToReadOnly(), and — critically — “writable regions cannot be created this way because their read-only descriptor cannot be created.” That is a compile-time encoding of exactly the property seals provide at runtime: a renderer must not retain a writable handle to a buffer the browser process is about to parse.

The recurring production lesson across all of them is that the shared layout must be an explicit, versioned contract: because there is no kernel-enforced schema, a mismatch between writer and reader struct layouts is a silent corruption, so production designs pin a magic number and version field at the head of every shared region and validate it on attach. A second, less obvious lesson is about accounting: shared pages are charged to whichever cgroup first faults them in, not to whoever “owns” the region, so a shared arena populated by a short-lived helper can leave a long-lived container holding the memory charge. And a third: shared memory is invisible to almost every observability tool that works at the syscall boundary. strace sees the mmap and nothing after it; there is no per-message tracepoint to attach to, no queue depth to graph. Designs that need visibility have to instrument themselves, typically with counters placed inside the shared region — which then need their own atomicity story.

See Also

• MAP_SHARED vs MAP_PRIVATE — the flag distinction that makes a mapping shared (this note) versus copy-on-write private
• Anonymous Shared Memory — the MAP_SHARED | MAP_ANONYMOUS fork-inheritance flavor in depth
• POSIX Shared Memory and shm_open — the named, /dev/shm-backed flavor
• memfd_create and Anonymous Memory Files — the modern anonymous-fd flavor in depth
• File Sealing with F_ADD_SEALS — the seal API, seal by seal
• Passing memfd Buffers Between Processes — the SCM_RIGHTS handoff, end to end
• Unix-Domain Sockets — fd passing, and the notification channel a shared region lacks
• Synchronizing Shared Memory with Futexes and Semaphores — the lock you must add; shared memory has none
• Futexes Across Process Boundaries — why PTHREAD_PROCESS_SHARED matters and how the shared futex path differs
• Acquire Release and Fence Semantics · Memory Barriers in the Linux Kernel — the ordering that MAP_SHARED does not give you
• mmap brk and Address Space System Calls — the address-space / page-table mechanics of mmap (not re-derived here)
• The Page Cache · The Page Cache and address_space — how file-backed shared pages live in the page cache and write back; the cache internals this note deliberately does not duplicate
• tmpfs In-Memory Filesystem — the filesystem behind /dev/shm and every memfd
• hugetlbfs and Reserved Huge Pages · Transparent Huge Pages — the two routes to huge-page-backed shared regions
• Anonymous vs File-Backed Memory — the broader anonymous-vs-file-backed distinction
• The Kernel Futex Interface — the kernel primitive behind cross-process locks
• Direct IO and O_DIRECT — the other way to take the kernel out of the data path, from the file-I/O side
• Linux IPC MOC — parent map of content