vault

The Device Mapper Framework

19 min read

The Device Mapper Framework

The device mapper (abbreviated dm) is the Linux kernel framework that builds virtual block devices by mapping ranges of a virtual device’s sector address space onto one or more targets — pluggable modules that each know how to transform or redirect the block-I/O operations (bio structures) that land on them. A dm device carries a mapping table: a list of rows, each row being start length target-type target-args, that partitions the virtual device’s sectors and says which target handles each range and how. When a bio arrives, dm looks up the target covering its starting sector, hands the bio to that target’s map method, and the target either remaps it to an underlying device, clones and splits it, or completes it itself. Everything Linux storage stacks atop — LVM, LUKS, dm-thin, multipath — is a device-mapper target. This note covers the generic dm machinery (pinned to Linux 6.12 LTS, released 2024-11-17; spot-checked against 6.18 LTS, 2025-11-30); the userspace volume manager that drives it lives in LVM Logical Volume Management.

Mental Model

The right way to think about dm is as a routing table for sectors, structurally identical to a network routing table but operating on a block device’s linear sector address space instead of an IP address space. A network router looks up a destination IP in a longest-prefix-match table and forwards the packet to the matching next-hop interface; dm looks up a bio’s starting sector in a sector-range table and forwards the bio to the matching target. The target is the next hop. Crucially, a dm target consumes a bio and emits bios downstream — exactly like the block layer’s other stacking devices — which is why dm targets compose: a dm-crypt device can sit on top of an LVM linear device which sits on top of an md RAID device, each layer remapping the bio and passing it down.

flowchart TB
  APP["File system / app<br/>submit_bio(bio)"] --> DMDEV["/dev/mapper/myvol<br/>(mapped_device)"]
  DMDEV --> SUB["dm_submit_bio()<br/>get live table"]
  SUB --> FIND["dm_table_find_target(sector)<br/>(b-tree lookup)"]
  FIND --> SPLIT["__split_and_process_bio()<br/>clip bio to target boundary"]
  SPLIT --> MAP["ti-&gt;type-&gt;map(ti, clone)"]
  MAP -->|"DM_MAPIO_REMAPPED"| DOWN["dm_submit_bio_remap()<br/>-&gt; underlying device"]
  MAP -->|"DM_MAPIO_SUBMITTED"| OWN["target completed it<br/>(e.g. dm-zero, error)"]
  DOWN --> REAL["/dev/sda, /dev/nvme0n1, ...<br/>(or another dm device)"]

The path of one bio through a device-mapper device. What it shows: a bio enters the virtual device, dm finds the target covering its starting sector via a b-tree, clips the bio so it does not cross a target boundary, and calls that target’s map method, which either remaps the bio to a real device (the common case) or completes it in place. The insight to take: dm is a thin dispatcher — all the intelligence is in the targets, and because each target emits bios downstream, dm devices stack arbitrarily.

The Core Abstractions: target_type and dm_target

Two structures carry the whole framework. A struct target_type describes a kind of target — the code — and a struct dm_target describes one instance of a target within a specific table — the data. The relationship is class-versus-object: there is exactly one target_type named "linear" registered in the kernel, but a system may have hundreds of dm_target instances all pointing at it, one per linear segment in every LVM volume.

The target_type (from include/linux/device-mapper.h) is a vtable of method pointers:

struct target_type {
	uint64_t features;
	const char *name;
	struct module *module;
	unsigned int version[3];
	dm_ctr_fn ctr;            /* constructor: parse args, build private state */
	dm_dtr_fn dtr;            /* destructor: tear it down */
	dm_map_fn map;            /* the hot path: handle one bio */
	dm_endio_fn end_io;       /* called when a remapped bio completes */
	dm_status_fn status;      /* report table / runtime status as text */
	dm_message_fn message;    /* receive a runtime control message */
	dm_prepare_ioctl_fn prepare_ioctl;
	dm_iterate_devices_fn iterate_devices;
	dm_io_hints_fn io_hints;
	/* ... and more: presuspend, postsuspend, preresume, resume, busy ... */
	struct list_head list;
};

The method signatures matter. The constructor is typedef int (*dm_ctr_fn)(struct dm_target *target, unsigned int argc, char **argv) — it receives the target arguments already tokenized into argv and must validate them, acquire any underlying devices, and stash per-instance state in target->private. The hot-path mapper is typedef int (*dm_map_fn)(struct dm_target *ti, struct bio *bio) — it inspects the bio, does whatever the target does, and returns one of the DM_MAPIO_* codes. The status method is typedef void (*dm_status_fn)(struct dm_target *ti, status_type_t status_type, unsigned int status_flags, char *result, unsigned int maxlen) — it serializes the target’s configuration (for STATUSTYPE_TABLE, used to reproduce the table) or runtime state (for STATUSTYPE_INFO) into a text buffer.

The per-instance struct dm_target records where this target lives in the address space and points back at its type:

struct dm_target {
	struct dm_table *table;
	struct target_type *type;
	sector_t begin;          /* first sector this target covers */
	sector_t len;            /* number of sectors it covers */
	uint32_t max_io_len;     /* targets can cap bio size here */
	unsigned int num_flush_bios;
	unsigned int num_discard_bios;
	void *private;           /* target's per-instance state */
	char *error;             /* ctr sets this on failure for userspace */
	bool discards_supported:1;
	bool flush_supported:1;
	/* ... */
};

begin and len are the row’s sector range; private is whatever the constructor allocated (for the linear target, a small struct holding the underlying device pointer and the offset). When the constructor fails, it writes a human-readable reason into ti->error, which propagates back to the dmsetup user.

How a target_type joins the kernel

Targets register themselves into a single global list. From drivers/md/dm-target.c:

static LIST_HEAD(_targets);
static DECLARE_RWSEM(_lock);
 
int dm_register_target(struct target_type *tt) {
	down_write(&_lock);
	if (__find_target_type(tt->name))
		rv = -EEXIST;
	else
		list_add(&tt->list, &_targets);
	up_write(&_lock);
	return rv;
}

When a table is loaded that names target "crypt", dm calls dm_get_target_type("crypt"), which scans _targets; if no match is found it triggers an on-demand module load (request_module) and retries — this is why loading a dm-crypt table auto-loads dm_crypt.ko. The lookup also takes a module reference (try_module_get) so the target module cannot be unloaded while a live table uses it.

The Mapping Table and Target Lookup

A mapping table is an ordered list of rows that tile the virtual device’s entire sector range with no gaps and no overlaps. Each row is start length target-type args. The table is built by dm_table_add_target(), which for each row: resolves the type name (ti->type = dm_get_target_type(type)), splits the argument string into argc/argv (dm_split_args), and calls the type’s constructor (r = ti->type->ctr(ti, argc, argv)). Once every row is added, dm_table_complete() finalizes the table, which includes building a search index.

That index is the key to the hot path. Rather than linearly scanning rows for every bio, dm builds a B-tree of the per-target high-sector boundaries (t->highs[] laid out as a multi-level tree in t->index[]). Lookup is dm_table_find_target():

struct dm_target *dm_table_find_target(struct dm_table *t, sector_t sector)
{
	unsigned int l, n = 0, k = 0;
	sector_t *node;
	if (unlikely(sector >= dm_table_get_size(t)))
		return NULL;
	for (l = 0; l < t->depth; l++) {
		n = get_child(n, k);
		node = get_node(t, l, n);
		for (k = 0; k < KEYS_PER_NODE; k++)
			if (node[k] >= sector)
				break;
	}
	return &t->targets[(KEYS_PER_NODE * n) + k];
}

The traversal descends t->depth levels, at each level finding the first key >= sector, which yields the target whose range contains the sector in O(log n) rather than O(n). For a table with one or two targets this is overkill; for an LVM volume composed of dozens of linear segments stitched across many physical volumes, it matters.

How a bio Flows Through dm

Every dm device registers dm_submit_bio as its ->submit_bio, so when the upper layer calls submit_bio() on /dev/mapper/myvol, control lands here. From drivers/md/dm.c:

static void dm_submit_bio(struct bio *bio)
{
	struct mapped_device *md = bio->bi_bdev->bd_disk->private_data;
	struct dm_table *map = dm_get_live_table(md, &srcu_idx);
 
	if (unlikely(test_bit(DMF_BLOCK_IO_FOR_SUSPEND, &md->flags))) {
		/* device is suspended: queue the bio for later */
		queue_io(md, bio);
		goto out;
	}
	dm_split_and_process_bio(md, map, bio);
	dm_put_live_table(md, srcu_idx);
}

Two design choices appear immediately. First, the live table is fetched under SRCU (sleepable read-copy-update), a read-mostly synchronization mechanism — bio submission is frequent and the table changes rarely, so readers must not contend. Second, if the device is suspended, the bio is queued, not failed — this is the mechanism that makes atomic table swaps possible (covered below).

The interesting work is in __split_and_process_bio, which clips the bio so it never straddles a target boundary:

static blk_status_t __split_and_process_bio(struct clone_info *ci)
{
	struct dm_target *ti = dm_table_find_target(ci->map, ci->sector);
	len = min_t(sector_t, max_io_len(ti, ci->sector), ci->sector_count);
	clone = alloc_tio(ci, ti, 0, &len, GFP_NOIO);
	__map_bio(clone);
	ci->sector += len;
	ci->sector_count -= len;
	return BLK_STS_OK;
}

It finds the target, computes len as the smaller of “sectors remaining in this bio” and “sectors until the next target boundary” (max_io_len), clones the bio so it spans only that length, and maps the clone. If the original bio crossed a boundary, the loop runs again for the remainder against the next target. This is why dm can present a single contiguous device backed by physically disjoint segments: each bio is silently split at segment edges.

Cloning rather than mutating the original bio is deliberate: dm tracks outstanding clones in a struct dm_io so it can aggregate their completions and report a single result upstream, and a target may need the original bio intact (for retry, or because one logical bio fans out to several targets — for example, a mirror writes the same data to two legs).

__map_bio makes the actual call into the target and acts on the return code:

static void __map_bio(struct bio *clone)
{
	struct dm_target *ti = clone_to_tio(clone)->ti;
	clone->bi_end_io = clone_endio;
	r = ti->type->map(ti, clone);
	switch (r) {
	case DM_MAPIO_SUBMITTED:        /* target took ownership; nothing to do */
		break;
	case DM_MAPIO_REMAPPED:         /* target rewrote dest; we submit it */
		dm_submit_bio_remap(clone, NULL);
		break;
	case DM_MAPIO_KILL:
	case DM_MAPIO_REQUEUE:
		free_tio(clone);
		dm_io_dec_pending(io, error);
		break;
	}
}

The two common return codes capture the two target idioms. DM_MAPIO_REMAPPED means “I’ve rewritten this bio’s destination device and sector; please submit it” — dm then calls dm_submit_bio_remap, which records I/O accounting and sends the bio down to the underlying block device. DM_MAPIO_SUBMITTED means “I’ve taken full responsibility for this bio; I will call bio_endio myself” — used by targets that complete the bio in place (dm-zero zero-fills and ends it) or that need to do asynchronous work before issuing it (dm-crypt queues the bio to a worker thread for encryption). DM_MAPIO_KILL fails the bio; DM_MAPIO_REQUEUE (and DM_MAPIO_DELAY_REQUEUE) asks the block layer to retry later. The full set is just five constants:

#define DM_MAPIO_SUBMITTED	0
#define DM_MAPIO_REMAPPED	1
#define DM_MAPIO_REQUEUE	2
#define DM_MAPIO_DELAY_REQUEUE	3
#define DM_MAPIO_KILL		4

One refinement: a target may want to handle only part of the clone (common in dm-thin, where a bio may span a provisioned and an unprovisioned block). dm_accept_partial_bio(bio, n_sectors) lets the target shrink the clone to n_sectors and have dm re-issue the rest as a follow-up bio.

A Worked Example: the linear Target

The simplest target is linear, which maps a virtual range onto a contiguous range of an underlying device. Its whole job is to add a fixed offset to each bio’s sector and redirect it. From drivers/md/dm-linear.c:

static struct target_type linear_target = {
	.name    = "linear",
	.version = {1, 4, 0},
	.ctr     = linear_ctr,
	.dtr     = linear_dtr,
	.map     = linear_map,
	.status  = linear_status,
	.prepare_ioctl = linear_prepare_ioctl,
};

The constructor parses exactly two arguments — the underlying device and the start offset:

static int linear_ctr(struct dm_target *ti, unsigned int argc, char **argv)
{
	if (argc != 2) { ti->error = "Invalid argument count"; return -EINVAL; }
	if (sscanf(argv[1], "%llu%c", &tmp, &dummy) != 1 || tmp != (sector_t)tmp)
		{ ti->error = "Invalid device sector"; ... }
	lc->start = tmp;
	dm_get_device(ti, argv[0], dm_table_get_mode(ti->table), &lc->dev);
	ti->private = lc;
	return 0;
}

dm_get_device resolves the device path (or major:minor) and takes a reference so the underlying device cannot vanish under a live mapping. The map method is almost trivially short:

static int linear_map(struct dm_target *ti, struct bio *bio)
{
	struct linear_c *lc = ti->private;
	bio_set_dev(bio, lc->dev->bdev);
	bio->bi_iter.bi_sector = linear_map_sector(ti, bio->bi_iter.bi_sector);
	return DM_MAPIO_REMAPPED;
}

linear_map_sector computes lc->start + dm_target_offset(ti, sector) — that is, it subtracts the target’s begin (to get the offset within this target) and adds the underlying-device start. bio_set_dev repoints the bio at the real device, and returning DM_MAPIO_REMAPPED tells dm to submit it. The status method, for STATUSTYPE_TABLE, emits "%s %llu" — device name and start — which is exactly the table line that would recreate it.

The striped target (dm-stripe.c) is the same idea with arithmetic: its arguments are <num_stripes> <chunk_size> [<dev> <offset>]+, and stripe_map_sector computes which device a sector lands on by chunk = sector / chunk_size; stripe = chunk % num_stripes (using bit-shifts when num_stripes is a power of two — sc->stripes_shift = __ffs(stripes)). That spreads sequential logical sectors round-robin across devices — RAID-0 in a single target.

A useful contrast is the zero target (dm-zero.c), which never touches an underlying device and returns DM_MAPIO_SUBMITTED:

static int zero_map(struct dm_target *ti, struct bio *bio)
{
	switch (bio_op(bio)) {
	case REQ_OP_READ:
		zero_fill_bio(bio);   /* reads return zeros */
		break;
	case REQ_OP_WRITE:
	case REQ_OP_DISCARD:
		break;                /* writes silently dropped */
	default:
		return DM_MAPIO_KILL;
	}
	bio_endio(bio);
	return DM_MAPIO_SUBMITTED;
}

And the error target is one line — io_err_map returns DM_MAPIO_KILL for everything, useful to plug a hole in an address space that must never be read.

The Catalog of Targets

The framework ships a large set of targets; the admin-guide index documents them. The ones most relevant here:

  • linear — contiguous remap onto a device at an offset. The building block of LVM linear LVs.
  • striped — RAID-0 round-robin across N devices by chunk. Backs LVM striped LVs.
  • error — fails all I/O. Used to mask bad ranges or test error paths.
  • zero — reads return zeros, writes are discarded. A bottomless sink/source.
  • delay — injects artificial latency; useful for testing timeout handling.
  • flakey — periodically drops/corrupts I/O to simulate flaky hardware in tests.
  • crypt — transparent encryption; see dm-crypt and LUKS Disk Encryption.
  • thin / thin-pool — copy-on-write thin provisioning and snapshots; see Thin Provisioning and Snapshots dm-thin.
  • snapshot / snapshot-origin — the older COW snapshot mechanism.
  • raid — wraps the md RAID personalities (see Software RAID with md) behind a dm target, which is how LVM does RAID.
  • integrity, verity, dust, ebs, clone, cache, writecache, multipath — integrity protection, read-only authentication (dm-verity, used by Android and Chrome OS for verified boot), bad-block simulation, caching, and path failover.

The Control Plane: the dm-ioctl Interface

Userspace never writes mapping tables through a filesystem — it drives dm through a single character device, /dev/mapper/control, using ioctl() calls defined in include/uapi/linux/dm-ioctl.h. The interface is versioned independently of the kernel; in 6.12 it reports protocol 4.48.0 (DM_VERSION_MAJOR 4, DM_VERSION_MINOR 48, dated -ioctl (2023-03-01)). Every call passes a struct dm_ioctl header naming the device, followed by command-specific payload:

struct dm_ioctl {
	__u32 version[3];
	__u32 data_size;       /* total ioctl buffer size */
	__u32 data_start;      /* where the payload begins */
	__u32 target_count;    /* number of dm_target_spec rows following */
	__s32 open_count;
	__u32 flags;           /* DM_SUSPEND_FLAG, DM_READONLY_FLAG, ... */
	__u64 dev;             /* device number, in/out */
	char  name[DM_NAME_LEN];
	char  uuid[DM_UUID_LEN];
	char  data[7];
};

For a table load, the payload is a sequence of struct dm_target_spec rows, each followed by its argument string:

struct dm_target_spec {
	__u64 sector_start;
	__u64 length;
	__s32 status;
	__u32 next;            /* byte offset to the next spec */
	char  target_type[DM_MAX_TYPE_NAME];
};

The principal commands (dm-ioctl.c dispatches them through a _ioctls[] table):

  • DM_DEV_CREATE — allocate a new empty mapped device, insert it in the name/UUID hash. The device exists but has no table; reads/writes block.
  • DM_TABLE_LOAD — build a table from the dm_target_spec rows and install it as the device’s inactive table. table_load calls populate_table, which loops dm_table_add_target(...) per spec, then dm_table_complete. Critically, this does not affect the running device.
  • DM_DEV_SUSPEND — with DM_SUSPEND_FLAG set, suspend the device (do_suspend → dm_suspend); with it clear, resume it (do_resume).
  • DM_DEV_REMOVE, DM_DEV_RENAME, DM_DEV_STATUS, DM_TABLE_STATUS, DM_TABLE_DEPS, DM_LIST_DEVICES, DM_TARGET_MSG (send a runtime message to a target, e.g. to a thin pool).

Atomic table swaps via suspend/resume

The most important property of dm is that you can replace a live device’s table without losing in-flight I/O. This is the mechanism behind LVM resizes, snapshot creation, pvmove, and live reconfiguration. It works because table load and table activation are separate steps:

  1. DM_TABLE_LOAD builds the new table and parks it as the inactive table (hc->new_map). The device keeps serving I/O from its active table.
  2. DM_DEV_SUSPEND (suspend) flips DMF_BLOCK_IO_FOR_SUSPEND, so dm_submit_bio starts queueing incoming bios instead of dispatching them, and waits for already-dispatched I/O to drain. The device is now quiescent.
  3. DM_DEV_SUSPEND (resume) calls dm_swap_table(md, new_map), which atomically makes the inactive table active and returns the old one, then dm_resume clears the suspend flag and replays the queued bios — but now against the new table.
/* do_resume(), simplified */
old_map = dm_swap_table(md, new_map);   /* inactive -> active, atomically */
if (dm_suspended_md(md))
	dm_resume(md);                  /* unblock + replay queued bios */

From the application’s perspective nothing happened except a brief stall; underneath, the entire sector-to-device mapping may have changed. Because the SRCU-protected live table pointer is swapped under lock and old readers drain via SRCU, no bio is ever dispatched against a half-built table.

A Worked dmsetup Example

dmsetup is the low-level userspace tool that speaks the ioctl protocol directly (LVM uses the same library, libdevmapper). A table line is logical_start_sector num_sectors target_type target_args (dmsetup(8)). To concatenate two disks into one virtual device:

# Create a device "joined" whose first part maps /dev/sdb and second maps /dev/sdc
echo "0           1953525168 linear /dev/sdb 0
      1953525168  1953525168 linear /dev/sdc 0" | dmsetup create joined

Line by line:

  • 0 1953525168 linear /dev/sdb 0 — sectors 0 through 1953525167 of /dev/mapper/joined map to the linear target, backed by /dev/sdb starting at its sector 0.
  • 1953525168 1953525168 linear /dev/sdc 0 — the next 1953525168 sectors map to /dev/sdc from its sector 0. The two rows tile the whole 3.9-billion-sector address space with no gap.

After this, /dev/mapper/joined is a single device twice the size of either disk; a bio to sector 0 goes to sdb, a bio to sector 2,000,000,000 goes to sdc (after dm subtracts the target’s begin). Inspect it:

dmsetup table joined      # prints the table back (STATUSTYPE_TABLE)
dmsetup status joined     # runtime status (STATUSTYPE_INFO)
dmsetup deps joined       # underlying devices: (8:16) (8:32)
dmsetup info joined       # state, open count, major:minor, table live/inactive

To grow it live, you would dmsetup load joined <new-table> (loads inactive) then dmsetup suspend joined; dmsetup resume joined (atomic swap) — exactly the sequence LVM automates for lvextend.

A three-way stripe (RAID-0) over three disks with a 64 KiB chunk (128 sectors of 512 bytes):

echo "0 5860515840 striped 3 128 /dev/sdb 0 /dev/sdc 0 /dev/sdd 0" \
  | dmsetup create stripe0

Here 3 is the stripe count, 128 the chunk size in sectors, then three device offset pairs. Sequential logical sectors now scatter across all three spindles in 64 KiB chunks.

Failure Modes and Diagnosis

  • device-mapper: table: 253:N: linear: Device lookup failed — a table referenced an underlying device that does not exist or is not readable. The constructor’s dm_get_device failed; the human-readable reason is in ti->error and echoed by dmsetup.
  • Tables that do not tile the whole device — if your rows leave a gap, the missing sectors return I/O errors when accessed (effectively the error target). dm does not require you to cover the whole device, but reads to uncovered ranges fail. dm_table_find_target returns NULL for sectors past the table size.
  • Suspend that never completes — dmsetup suspend blocks until in-flight I/O drains. If the underlying device has hung I/O (a failing disk, a stuck NFS-backed loop device), the suspend hangs, and because the active table cannot be swapped, any operation needing a suspend (snapshot, resize) hangs too. This is a classic cause of a wedged LVM operation.
  • Module not present — loading a crypt or thin-pool table on a kernel without the module returns -EINVAL with unknown target type; dm_get_target_type’s auto-request_module only works if the module is installed.
  • Stale /dev/mapper nodes — dm device nodes are created by udev reacting to uevents, not by the kernel directly. If udev is not running (early boot, broken initramfs), dmsetup --noudevrules ... mknodes may be needed to materialize the nodes.

Uncertain

Verify: the exact set and ordering of steps dm_suspend performs to quiesce I/O (flush vs. no-flush behavior under DM_NOFLUSH_FLAG, and whether it waits on the md/SRCU drain before or after blocking new submissions). Reason: reconstructed from the dm-ioctl.c handler summaries and dm.c snippets rather than a line-by-line read of dm_suspend/__dm_suspend. To resolve: read dm_suspend and __dm_suspend in drivers/md/dm.c v6.12 in full. uncertain

Alternatives and When to Choose Them

The device mapper is one of three ways the kernel composes block devices, and they overlap:

  • Software RAID via md (see Software RAID with md) predates dm and implements RAID-0/1/5/6/10 as its own stacking layer with /dev/mdN devices and superblocks. dm has a raid target that actually wraps the md RAID engine, so LVM RAID and mdadm RAID share kernel code but differ in metadata and management. Choose md/mdadm for a pure, self-describing RAID array; choose dm/LVM RAID when you want RAID integrated with volume management.
  • The loop device and the block-layer’s own splitting handle file-backed devices and request-size limits without dm — dm is overkill if you only need to expose a file as a block device (losetup).
  • Btrfs/ZFS fold volume management, RAID, snapshots, and checksumming into the filesystem, bypassing dm entirely. They trade dm’s composability and filesystem-agnosticism for tighter integration and end-to-end checksums.

dm’s distinguishing strength is composability and filesystem-agnosticism: any target stacks on any other, under any filesystem, with atomic live reconfiguration. That is why it is the substrate for LVM, LUKS, multipath, and verified boot, even though specialized tools beat it on their own turf.

Production Notes

In practice you rarely invoke dmsetup by hand — LVM, cryptsetup, and multipathd drive the ioctl interface through libdevmapper and present friendlier abstractions. But dmsetup table, dmsetup status, dmsetup ls --tree, and dmsetup deps are indispensable for seeing what is actually mapped, especially when LVM’s view and the kernel’s view diverge (e.g. after a botched pvmove or a snapshot that filled up). The device numbers you see are major 253 (or 254) — the dm major — and /sys/block/dm-N/ exposes each device’s queue and slaves/holders symlinks that reveal the stacking graph. When debugging a layered stack (LUKS over LVM over md), reading the dm tables top-down is the fastest way to find where a bio would actually go.

See Also

Graph View

Backlinks