vault

Btrfs Copy-on-Write Subvolumes and Snapshots

16 min read

Btrfs Copy-on-Write Subvolumes and Snapshots

The defining behavior of Btrfs is copy-on-write (CoW): once the transaction that allocated a block has committed, Btrfs never overwrites that block in place. Any later write to that logical location goes to a newly allocated block, and the pointers above it — in the B-trees and ultimately the superblock — are updated to point at the new location (Btrfs design doc). This one rule buys three things at once: crash consistency without overwriting live data (a crash mid-write leaves the old committed tree fully intact, because the superblock still points at it until the new tree is durable), cheap writable snapshots (share a tree root, diverge only on write), and reflinks (cheap file clones that share extents). This note explains the CoW transaction mechanism and the subvolume/snapshot/reflink user model it enables; the on-disk B-tree forest those snapshots live in is the subject of its sibling Btrfs Filesystem Internals.

Version

Pinned to Linux 6.12 LTS (2024-11-17). Where a behavior changed across versions (notably cross-mount reflink), the version is called out inline.

Mental Model: Shadow the Tree Upward, Then Flip One Pointer

CoW in Btrfs is the “shadowing” technique from Ohad Rodeh’s B-tree research. To modify a leaf, you do not edit it — you write a new copy of the leaf at a fresh location, then write a new copy of its parent node (which now points at the new leaf), and so on up to the tree root. The path from the changed leaf to the root is duplicated; every block not on that path is still shared and untouched. The whole edit becomes visible atomically the moment a single pointer at the very top — in the superblock — is updated to the new root.

flowchart TB
  subgraph OLD["Before commit (live, on disk)"]
    SBo["Superblock<br/>points at root R"]
    SBo --> R["root R"]
    R --> A["node A"] --> L1["leaf L1"]
    R --> B["node B"]
    B --> L0["leaf L0"]
    B --> L2["leaf L2 (to be changed)"]
  end
  subgraph NEW["After commit"]
    SBn["Superblock<br/>now points at root R'"]
    SBn --> Rp["root R' (new)"]
    Rp --> A2["node A (SHARED, unchanged)"]
    Rp --> Bp["node B' (new)"]
    A2 --> L1
    Bp --> L0b["leaf L0<br/>(SHARED)"]
    Bp --> L2p["leaf L2' (new, changed copy)"]
  end

CoW as upward shadowing. What it shows: changing one leaf (L2) re-writes only the blocks on the root-to-leaf path — the new leaf L2', its parent B', and a new root R' — while every off-path block is shared with the old tree: the whole node A subtree (including leaf L1) is unchanged, and even B'’s sibling leaf L0 is the same block as before. The commit is the single superblock pointer flip from R to R'. The insight: until that flip lands durably, the old tree R is wholly intact and is what a crash recovers to; consistency comes “for free” from never mutating committed blocks plus one atomic pointer write — no undo/redo journal of data needed. The same shared structure means a snapshot is just “keep R around” — it costs one extra root pointer and a refcount bump.

Two different CoWs — don't conflate them

This is filesystem-block CoW: it shares and clones on-disk extents and B-tree blocks, made atomic by re-pointing a tree and flipping the superblock. It is conceptually the same “share until someone writes” idea as anonymous-page CoW at fork() — see Copy-on-Write and fork — but at a different layer and with different enforcement. At fork(), the parent and child share physical RAM pages marked read-only in the page tables, and the MMU raises a page fault on the first write, at which point the kernel copies that one page. In Btrfs, there is no hardware trap; the filesystem code itself allocates a new block on write and rewires the tree, and the unit of sharing is a disk extent, not a 4 KiB RAM page. Both are “lazy copy,” one enforced by the CPU’s memory-management unit, the other by the filesystem’s write path.

The CoW Transaction Commit, Step by Step

CoW only protects blocks once their allocating transaction has committed. Within an open transaction, the first write to a block can still go in place (it has not been promised to anyone yet); after commit, that block is frozen and the next write shadows it (design doc). A commit proceeds roughly as follows:

  1. Shadow every dirtied path. For each B-tree touched this transaction, the changed leaves and the nodes above them up to each tree’s root are written to newly allocated blocks. Off-path blocks remain shared.
  2. Handle reference counting for refcounted trees. All subvolume (FS) trees are reference-counted. When a node is CoWed, the refcount of every block it points to is incremented (because the old tree still points at them too); for a leaf, the refcounts of the file extents it references are incremented (design doc). The root tree and extent tree are not refcounted — they are protected by CoW logging alone, and their replaced blocks are freed (but not reused) until the freeing transaction commits.
  3. Update the tree of tree roots. Each new subvolume root is recorded with a new ROOT_ITEM keyed (subvolume objectid, BTRFS_ROOT_ITEM_KEY, transaction id). Briefly, the root tree holds two pointers per changed subvolume — the new root and the one from the previous transaction.
  4. Flush all new B-tree blocks to disk, then write the new superblock pointing at the new root tree. “Once the super block has been properly written to disk, the transaction is considered complete” (design doc). The superblock write is the atomic commit point.
  5. Reclaim later. After commit, the old subvolume root items may be removed and their root blocks’ refcounts lowered by one; a block whose count hits zero is freed, recursively freeing what it pointed at in a depth-first walk. For a huge tree this is done in pieces, recording a progress key in the root tree so the work can span transactions and resume safely after a crash (design doc).

Consistency vs durability — and the log tree

The CoW + atomic-superblock mechanism above gives crash consistency: a crash before the new superblock lands recovers the old, fully-valid tree; a crash after recovers the new one. It does not by itself give low-latency fsync durability, because a commit is a heavyweight whole-filesystem operation. For that, Btrfs has a separate log tree (BTRFS_TREE_LOG_OBJECTID) — a small write-ahead log of just the fsync’d changes, replayed on the next mount (btrfs_tree.h comment: “does write ahead logging to speed up fsyncs”). So the common shorthand “Btrfs has no journal” is half-right: it has no metadata/data journal for consistency (CoW handles that), but it does keep a log tree purely as an fsync optimization. The journaling-vs-CoW comparison lives in Crash Consistency and fsck; the on-disk role of the log tree is catalogued in Btrfs Filesystem Internals.

Subvolumes — Independent Roots in One Filesystem

A subvolume is “a part of [the] filesystem with its own independent file/directory hierarchy and inode number namespace” (subvolumes doc). Concretely, it is a separate FS B-tree with its own root, named in the tree of tree roots — independent directory tree, independent inode-number space, but sharing the single storage pool of the whole filesystem. Unlike an LVM logical volume (a block-level partition with a fixed size), a Btrfs subvolume is extent-based and elastic: subvolumes can freely share file extents with each other.

Key facts:

  • The subvolume root directory always has inode number 256 (sometimes called “the root of the subvolume”) (subvolumes doc). This means inode numbers are not unique across a Btrfs filesystem — every subvolume reuses 256 for its top. Applications needing a true unique id must use the (subvolume id, inode number) pair (read the subvolume/root id via btrfs inspect-internal rootid or the BTRFS_IOC_INO_LOOKUP ioctl).
  • The top-level subvolume has id 5 and is created with the filesystem; it cannot be removed or replaced, and is mounted by default unless a different default is set with btrfs subvolume set-default.
  • A subvolume can be accessed two ways: as an ordinary directory reachable from its parent, or as a separately mounted filesystem via the subvol= / subvolid= mount options — in which case the parent is hidden, much like a bind mount (and in fact the subvolume mount is implemented as a bind mount under the hood) (subvolumes doc).
  • The subvolume/rootid is persistent and immutable. You can rename or move a subvolume, but its numeric id never changes.

Up to 2^64 subvolumes may exist on one filesystem (design doc).

Snapshots — CoW Clones of a Subvolume

A snapshot is just a subvolume with a given initial content — that content being the state of another subvolume at the moment of snapshotting (subvolumes doc). Mechanically: “Snapshots are identical to subvolumes, but their root block is initially shared with another subvolume. When the snapshot is taken, the reference count on the root block is increased, and the copy on write transaction system ensures changes made in either the snapshot or the source subvolume are private to that root” (design doc).

That is the whole trick. Creating a snapshot:

  1. adds a new ROOT_ITEM in the tree of tree roots that points at the same root block as the source subvolume, and
  2. bumps that root block’s reference count by one.

Nothing else is copied. Per the subvolumes doc: once any pending dirty data is flushed, “the snapshot is instantaneous and only creates a new tree root copy in the metadata.” From then on, the source and snapshot are two FS trees that share every block; the first write to either side CoWs the affected path (per the transaction commit above), incrementing/decrementing refcounts so the two diverge only on the blocks each actually changes. A 100 GiB subvolume snapshotted costs essentially nothing in space until you start writing — the snapshot’s footprint grows only with the delta between it and its origin.

Snapshots are writable by default. A plain snapshot is read-write; modifications in it do not affect the origin and vice versa. Passing -r makes the snapshot read-only — and read-only snapshots are the building block of incremental send/receive (see Btrfs Filesystem Internals), because the receive side relies on the snapshot being unchanged to compute correct deltas. A read-only snapshot has its “last change generation” equal to its creation generation; flipping a received read-only snapshot back to read-write resets the received_uuid and can break the incremental-send chain, so btrfs property set requires force to do it (subvolumes doc).

Snapshotting is not recursive — the nested-subvolume barrier

A subtle and frequently-surprising rule: a subvolume acts as a snapshotting barrier. If subvolume A contains a nested subvolume B, snapshotting A does not descend into B. Instead, the snapshot gets a stub (empty) subvolume with B’s name but inode number 2 and no contents (subvolumes doc):

$ btrfs subvolume create subvol1
$ btrfs subvolume create subvol1/subvol2     # nested subvolume
$ btrfs subvolume snapshot subvol1 snap1
$ find -ls
  256 ... ./subvol1                # real subvolume, inode 256
  256 ... ./subvol1/subvol2        # nested, inode 256
  257 ... ./subvol1/subvol2/file
  256 ... ./snap1                  # the snapshot, inode 256
    2 ... ./snap1/subvol2          # STUB — inode 2, empty, no ./file

The snapshot snap1 contains a subvol2 entry, but it is an empty stub (inode 2) — file is not present. (The stub is also not transmitted by send, so it is not recreated on receive.) This is why flat subvolume layouts are usually preferred over deeply nested ones, and why a “snapshot of /” needs separate subvolumes for /var, /var/log, etc., to control exactly what is and isn’t captured.

Deletion is two-phase

Deleting a subvolume/snapshot first removes its directory entry and queues the subvolume for cleaning; a background pass then walks and frees its no-longer-shared blocks one by one (subvolumes doc). Cleaning can take noticeable time proportional to the amount of unshared metadata, and several queued deletions process serially — so “I deleted a big snapshot but space didn’t come back instantly” is expected behavior, not a bug.

# Create a subvolume (a new, independent FS-tree root)
btrfs subvolume create /mnt/data
 
# Snapshot it (writable by default) — instantaneous, shares all blocks
btrfs subvolume snapshot /mnt/data /mnt/data-snap
 
# Read-only snapshot, the basis for send/receive
btrfs subvolume snapshot -r /mnt/data /mnt/data-snap-ro
 
# Inspect: id, parent, read-only flag, received_uuid, generations
btrfs subvolume show /mnt/data-snap-ro
 
# Delete (queued + cleaned in background)
btrfs subvolume delete /mnt/data-snap

The exact snapshot syntax (from btrfs-subvolume(8)) is btrfs subvolume snapshot [-r] [-i <qgroupid>] <source> <dest>|[<dest>/]<name> — -r makes the new snapshot read-only; -i adds it to a qgroup. If only dest is given, the snapshot is named after the basename of source; if source is not a subvolume, the command errors.

Reflink generalizes the same block-sharing to files. A reflink is “a type of shallow copy of file data that shares the blocks but otherwise the files are independent” — it builds directly on the underlying CoW mechanism, creating only new metadata pointing at the shared extents, much faster than a deep copy (reflink doc):

cp --reflink=always source target   # share extents; copy nothing until a write diverges them

Under the hood this is exactly the shared-extent trick from Btrfs Filesystem Internals: two files’ EXTENT_DATA items point into one on-disk extent (disk_bytenr), and the first write to either file CoWs only the touched range. Constraints (reflink doc):

  • No cross-filesystem reflink — there is no shared on-disk extent to point at between two filesystems.
  • Cross-mount-point reflink (two mounts of the same filesystem, e.g. two subvolume mounts) failed with “Cross device link” until kernel 5.17; it works since 5.18. On 6.12 it works.
  • Source and target must agree on NOCOW/checksum status. Reflinking a nodatacow (chattr +C) file to a normal file fails unless the target is also created NOCOW — because the shared extent’s checksum/CoW semantics must be consistent.

The Fragmentation and nodatacow Tradeoff

CoW’s cost is fragmentation. Because in-place overwrite is forbidden, a workload of small random writes into a large file — databases (PostgreSQL, MySQL/InnoDB), VM disk images, virtual-machine .qcow2, large append-mostly logs rewritten in place — scatters the file’s data across many small extents over time, degrading sequential read throughput and bloating the extent metadata (design doc on the three-way extent split per overwrite). Three knobs address it:

  • autodefrag (mount option, since kernel 3.0) — detects small random writes (currently ≤64 KiB ranges) and queues the regions for background defragmentation, merging them into fewer contiguous extents at the cost of some read latency and rewrite churn (btrfs(5)). Not recommended for large database workloads.
  • btrfs filesystem defrag — explicit, on-demand defragmentation; can also rewrite a file with compression.
  • nodatacow / chattr +C — disable CoW for specific files (or a whole mount) so they are updated in place, like a traditional filesystem. This restores good random-write performance for databases and VM images.

nodatacow is a real safety/feature tradeoff, not a free win

Per btrfs(5): nodatacow implies nodatasum and disables compression, and all files created under nodatacow get the NOCOW (C) file attribute. A NOCOW file therefore has no data checksums (so no scrub verification, no auto-repair-on-read of its data) and no compression, and its in-place updates reintroduce the possibility of torn/partial writes on a crash that CoW was avoiding. The C attribute itself describes the behavior precisely: “no copy-on-write, file data modifications are done in-place” (btrfs(5)). Per that same page, due to implementation limitations the C flag can be set/unset only on empty files — which is why the swapfile recipe does truncate -s 0 swapfile then chattr +C swapfile; when set on a directory, all newly created files inherit it. Set NOCOW only for files you have deliberately decided to forgo checksums and compression on — typically databases and VM images — and ideally keep those on a dedicated subvolume.

One subtlety the man page does not spell out: a NOCOW file that is shared by a snapshot must still be copied once on the first write after the snapshot (a single forced CoW, to preserve the snapshot’s view), after which it reverts to in-place updates. [!warning] Uncertain

Verify: the “NOCOW file is force-CoWed once on the first write after a snapshot, then reverts to in-place” claim. Reason: it is correct to my knowledge and consistent with how snapshots must preserve shared data, but it is not stated in the btrfs(5) text fetched here. To resolve: confirm against the kernel fs/btrfs NOCOW write path (e.g. run_delalloc_nocow / can_nocow_extent) at v6.12, or the btrfs wiki’s NOCOW notes. uncertain

Files touched by fallocate() are also implicitly excepted from compression (and preallocated), because a successful fallocate must guarantee future writes won’t ENOSPC, which is hard to promise in a CoW filesystem (compression doc).

Failure Modes and Common Misunderstandings

  • “A snapshot is a backup.” No — see Btrfs Filesystem Internals: a snapshot shares physical blocks with its origin, so media damage hits both. Snapshots guard against logical mistakes (bad upgrade, rm -rf), not bad sectors or bad RAM. Use redundant RAID profiles plus off-device send/receive for real backups.
  • “Deleting a snapshot frees space immediately.” No — deletion queues a background cleaning pass; space returns gradually as unshared blocks are freed.
  • “Inode numbers are unique on the filesystem.” No — every subvolume’s root is inode 256, and nested-snapshot stubs are inode 2. Use (subvolid, inode) for a unique identifier.
  • “cp --reflink copied my data twice.” No — it shared the extents; du may show the apparent size, but on-disk usage barely moved until you write.
  • NOCOW silently disabling checksums. Teams sometimes chattr +C a database directory for performance and are surprised that scrub no longer protects those files — checksums went away with CoW.

See Also

Graph View

Backlinks