switch_root and pivot_root at Boot

Once early userspace has found and mounted the real root filesystem somewhere off to the side, the last thing the initramfs /init does is pivot the machine onto that root and hand off to the real init — all without ever leaving PID 1. On a modern system this is the job of switch_root(8), the util-linux tool that “moves already mounted /proc, /dev, /sys and /run to newroot and makes newroot the new root filesystem and starts init process,” and which “removes recursively all files and directories on the current root filesystem” to give the RAM back (switch_root(8)). It is deliberately not the pivot_root(2) syscall: the initramfs is rootfs, the kernel’s special initial RAM filesystem, and “the rootfs (initial ramfs) cannot be pivot_root()ed” (pivot_root(2)). The two mechanisms solve the same problem — “change what / means and run the real init” — but switch_root destroys-and-overmounts the throwaway RAM root while pivot_root relocates and keeps the old root. This note is the boot angle: the mechanics of the pivot, why initramfs needs switch_root specifically, and why the legacy initrd used pivot_root instead. The syscall’s container-isolation semantics live in pivot_root and Changing the Root — this note cross-links it rather than re-deriving it.

This note is pinned to util-linux 2.40 and Linux 6.12 LTS (released 2024-11-17). The switch_root algorithm has been stable for over a decade; the kernel rootfs special-casing predates the 2.6 series.

Mental Model

The whole pivot exists because of one structural fact about Linux boot: the kernel mounts a special filesystem called rootfs (a ramfs, or tmpfs if configured) as / very early, unpacks the bootloader-supplied initramfs cpio archive into it (ramfs-rootfs-initramfs.rst), and then runs /init from it. That rootfs is permanent and unmountable — for the same reason PID 1 can’t be killed. The kernel documentation is blunt: “You can’t unmount rootfs for approximately the same reason you can’t kill the init process; rather than having special code to check for and handle an empty list, it’s smaller and simpler for the kernel to just make sure certain lists can’t become empty” (ramfs-rootfs-initramfs.rst).

So you cannot do the elegant thing — mount the real root, pivot_root onto it, and umount the old root to reclaim RAM — because rootfs refuses to be either pivoted or unmounted. The workaround is brute force: delete every file in rootfs to free the RAM, overmount rootfs with the real root via MS_MOVE, chroot into it, and exec the real init. That five-step dance is fiddly and dangerous to do by hand (a stray rm that crosses into the real root would be catastrophic), so it is packaged into the switch_root tool.

flowchart TB
  subgraph before["Before switch_root (in the initramfs)"]
    RF["/ = rootfs (ramfs/tmpfs)<br/>initramfs tools, PID 1 = /init"]
    SR["/sysroot or /newroot<br/>= real root, already mounted"]
    API["/proc /sys /dev /run<br/>(mounted on rootfs)"]
    RF --> SR
    RF --> API
  end
  before -->|"1. MS_MOVE /proc /sys /dev /run into newroot<br/>2. chdir newroot<br/>3. MS_MOVE newroot onto /<br/>4. chroot . ; chdir /<br/>5. fork: child rm -rf the old ramfs<br/>6. execv real /sbin/init"| after
  subgraph after["After switch_root (in the real root)"]
    NR["/ = real root filesystem<br/>PID 1 = real /sbin/init (same PID 1!)"]
    API2["/proc /sys /dev /run<br/>(moved over intact)"]
    NR --> API2
  end

The boot pivot performed by switch_root(8). What it shows: the API and tmpfs mounts (/proc, /sys, /dev, /run) are moved — not remounted — across the boundary so their state survives, then the new root is moved onto /, the old RAM root is deleted to reclaim memory, and the real init is exec’d. The insight to take: PID 1 never changes across the pivot — execv replaces the program image of the same process — and the old root is deleted, not unmounted, because rootfs cannot be unmounted.

Mechanical Walk-through — what switch_root actually does

The ground truth is the util-linux source, sys-utils/switch_root.c (v2.40). The interesting work is in two functions, switchroot() and recursiveRemove(), plus main(). Tracing it in order:

Step 0 — argument parsing (main). Usage is switch_root <newrootdir> <init> <args to init>. main() stores newroot = argv[1], init = argv[2], and initargs = &argv[2] (so argv[0] of the new init becomes the init path itself). It rejects empty arguments and demands at least two. Then it calls switchroot(newroot); only if that succeeds does it execv(init, initargs).

Step 1 — record the old root’s device id. switchroot() opens by stat("/", &oldroot_stat) and stat(newroot, &newroot_stat). These device IDs (st_dev) are the safety rails for everything that follows: the code uses them to tell “is this thing actually a separate mount, or just a directory on the same filesystem?” Comparing st_dev is how Unix tools detect mount-point crossings without parsing /proc/mounts.

Step 2 — move the API and runtime mounts. The list is hard-coded: const char *umounts[] = { "/dev", "/proc", "/sys", "/run", NULL };. For each, the code builds the destination path newroot + "/dev" etc., then:

• It skips the mount if stat("/dev", &sb) shows sb.st_dev == oldroot_stat.st_dev — meaning /dev is not a separate mount, just a plain directory on the old root, so there is nothing to move.
• It skips (and force-detaches) if the destination already has something mounted (sb.st_dev != newroot_stat.st_dev), via umount2(umounts[i], MNT_DETACH).
• Otherwise it does the real work: mount(umounts[i], newmount, NULL, MS_MOVE, NULL). MS_MOVE is the mount-relocation flag — it detaches a live mount and re-grafts it elsewhere atomically, without unmounting it, so every open file descriptor, the udev database, the journal in /run, the device nodes in /dev survive the move. If the move fails it warns and falls back to umount2(umounts[i], MNT_FORCE).

Why move these four specifically? Because they are virtual or tmpfs filesystems whose contents matter across the boundary. /run is a tmpfs that early userspace has been writing to (the saved cmdline, lock files, the systemd journal-in-initramfs); /dev is the devtmpfs with all the device nodes udev created; /proc and /sys are the kernel interfaces every subsequent program needs. Re-mounting them fresh would lose /run’s contents and momentarily blind userspace.

Step 3 — move the real root onto /. chdir(newroot) makes the real root the working directory. Then cfd = open("/", O_RDONLY) grabs a file descriptor on the old root before it disappears (this is how the delete step later reaches the old root). The pivot itself is mount(newroot, "/", NULL, MS_MOVE, NULL) — MS_MOVE again, this time grafting the real root mount directly over /. This is the move-and-overmount the kernel docs prescribe.

Step 4 — chroot and chdir into the new world. chroot(".") resets the process’s root to the (now overmounted) /, and chdir("/") repositions the working directory. After this, path resolution for the process happens entirely within the real root.

Step 5 — fork, then delete the old RAM root in the child. This is the subtle part the man page summarizes as the “remove recursively” warning. The code does:

switch (fork()) {
case 0: /* child */
    if (fstatfs(cfd, &stfs) == 0 &&
        (F_TYPE_EQUAL(stfs.f_type, STATFS_RAMFS_MAGIC) ||
         F_TYPE_EQUAL(stfs.f_type, STATFS_TMPFS_MAGIC)))
        recursiveRemove(cfd);
    else {
        warn(_("old root filesystem is not an initramfs"));
        close(cfd);
    }
    exit(EXIT_SUCCESS);
...
default: /* parent */
    close(cfd);
    return 0;
}

Two crucial guards here. First, the fstatfs check is the real safety gate: the child only deletes if the old root (cfd) is genuinely a ramfs or tmpfs (STATFS_RAMFS_MAGIC / STATFS_TMPFS_MAGIC). If you point switch_root at something that is not an in-memory throwaway root, it refuses to delete and just warns — this is what makes the “delete everything” behavior safe in exactly the boot case and inert otherwise. Second, the delete runs in a forked child while the parent returns immediately so main() can execv the real init; the child reaps the RAM in the background. (recursiveRemove closes cfd itself, so the parent only closes in the non-deleting branches.)

Step 6 — recursiveRemove(). It fdopendir(fd)s the old root, fstats it to capture the root’s st_dev (rb.st_dev), then walks entries. The load-bearing guard is if (sb.st_dev != rb.st_dev) continue; — it never crosses a mount point. It cannot accidentally descend into the moved-away /proc or into the real root, because those have different device IDs. For directories it openats and recurses, then unlinkat(..., AT_REMOVEDIR); for files it unlinkat(..., 0). This is the in-C equivalent of find / -xdev -delete, and the -xdev (don’t-cross-filesystems) behavior is exactly the st_dev comparison.

Step 7 — execv the real init (back in main). After switchroot() returns 0, main checks access(init, X_OK) (warns but proceeds — useful diagnostic if the init path is wrong) and then execv(init, initargs). execv replaces the program image of the current process; the PID does not change. That is the entire reason switch_root is run by the initramfs /init (PID 1) rather than spawned as a child: the real systemd or sysvinit must inherit PID 1. If execv returns at all, it failed, and errexec(init) aborts.

How the kernel reaches /init in the first place

switch_root is the end of early userspace; the kernel reaching /init is the start. From init/main.c (v6.12), kernel_init() first tries the unpacked initramfs: static char *ramdisk_execute_command = "/init"; and if present, run_init_process(ramdisk_execute_command). If there is no /init (no initramfs, or a broken one), it falls through to init= (the execute_command from the kernel command line) and then the classic fallback chain:

if (!try_to_run_init_process("/sbin/init") ||
    !try_to_run_init_process("/etc/init") ||
    !try_to_run_init_process("/bin/init") ||
    !try_to_run_init_process("/bin/sh"))
    return 0;
panic("No working init found.  Try passing init= option to kernel. ...");

The cpio archive is unpacked into rootfs by unpack_to_rootfs() (called from do_populate_rootfs()), and the compressed source memory is reclaimed afterward via free_initrd_mem() (init/initramfs.c). So the lineage is: kernel unpacks initramfs into rootfs → runs /init (PID 1) → /init mounts real root at /sysroot → /init execs switch_root /sysroot /sbin/init → real /sbin/init is PID 1. See Loading the Kernel and initramfs into Memory for the load step and The Early Userspace init Script for what /init does in between.

The legacy contrast — why initrd used pivot_root

Before initramfs (the cpio-into-rootfs model), Linux used initrd: a block-device RAM disk (/dev/ram0) holding a real filesystem image (ext2, etc.). The boot flow was different (initrd.rst): the bootloader loaded the image, the kernel turned it into a RAM disk and mounted it as root — but it was an ordinary mount on a block device, not the unmountable rootfs. So initrd’s /linuxrc or init could legitimately:

1. mount the real root on a subdirectory,
2. mkdir initrd for the old root,
3. pivot_root . initrd — “moves the current root to a directory under the new root, and puts the new root at its place,”
4. exec chroot . /sbin/init,
5. and later umount /initrd to reclaim the RAM disk and blockdev --flushbufs /dev/ram0.

The kernel doc states the key property plainly: “Note that changing the root directory does not involve unmounting it. It is therefore possible to leave processes running on initrd during that procedure” (initrd.rst). pivot_root keeps the old root — that is its whole point, and exactly why containers use it (you keep the host root mounted at put_old so you can later detach it cleanly; see pivot_root and Changing the Root).

The summary table that actually matters:

	initrd (legacy)	initramfs (modern)
Old root is	a block-device RAM disk, ordinary mount	rootfs — special, unmountable
Pivot mechanism	pivot_root(2) then umount	switch_root (MS_MOVE + delete + chroot)
Old root afterward	kept at put_old, then unmounted to free RAM	deleted in place to free RAM
Why	a real mount can be pivoted and unmounted	rootfs can be neither pivoted nor unmounted

This is why the ramfs-rootfs-initramfs.rst doc says of initramfs: “But initramfs is rootfs: you can neither pivot_root rootfs, nor unmount it. Instead delete everything out of rootfs to free up the space … overmount rootfs with the new root … and exec the new init” — and notes this is “remarkably persnickety,” which is precisely why the switch_root helper exists.

Configuration / Invocation

A hand-rolled initramfs /init (busybox-style) ends like this:

#!/bin/sh
# ... earlier: load modules, assemble RAID, unlock LUKS, find root ...
mount -o ro "$rootdev" /sysroot          # mount the real root off to the side
# move the kernel API filesystems we set up onto the new root:
# (switch_root will move /proc /sys /dev /run for us, so we don't here)
exec switch_root /sysroot /sbin/init     # PID 1 stays PID 1; never returns

Line-by-line: /sysroot is the conventional mount point for the real root inside an initramfs (systemd hard-codes this name; see below). exec is mandatory — without it, switch_root would run as a child of /init, the execv inside it would replace the child’s image, and PID 1 would still be the now-blocked /init. With exec, the shell’s process image is replaced by switch_root, which itself execvs /sbin/init, so the real init inherits PID 1 directly.

The mount-point requirement. switch_root “will fail to function if newroot is not the root of a mount” (switch_root.8.adoc). This is the st_dev logic from Step 3 — if /sysroot were just a directory on the initramfs (not a separate mount), moving it onto / would be meaningless. The man page gives the escape hatch: “you can first use a bind-mounting trick to turn any directory into a mount point” via mount --bind $DIR $DIR. In practice the real root is always a genuine mount, so this rarely bites.

The systemd-in-initramfs path. When the initramfs runs systemd (detected by /etc/initrd-release, default target initrd.target), the pivot is wrapped in units instead of a shell exec (bootup(7), systemd bootup.xml v257). The real root mounts at /sysroot (the sysroot.mount unit), initrd-root-fs.target and then initrd-fs.target are reached, initrd-cleanup.service isolates to initrd-switch-root.target, and finally initrd-switch-root.service “will cause the system to switch its root to /sysroot” and hands control to the host’s systemd. That service ultimately drives the same switch_root-equivalent kernel operations; the systemd-internal call is systemd-switch-root. See Early vs Late Userspace for the target-by-target ordering and systemd Targets and the Boot Sequence for the post-pivot targets.

The reverse pivot — /run/initramfs at shutdown

There is a mirror-image of the boot pivot at shutdown, and it explains why switch_root carefully moves /run over intact. The real root cannot be cleanly unmounted by a PID 1 that is itself running from the real root (you would be sawing off the branch you sit on). So systemd performs an “exitrd” pivot in reverse: “When the system manager is shutting down and /run/initramfs/shutdown exists, it will switch root to /run/initramfs/ and execute /shutdown” (systemd bootup.xml v257). Because /run is a tmpfs that was moved across the boot pivot (Step 2) and survives in RAM, the initramfs can stash a copy of itself at /run/initramfs during boot; at shutdown, systemd pivots back into that RAM image, which — running entirely from tmpfs, depending on nothing in the real root — can finally umount the real root and flush it. This is the symmetric bookend to switch_root: boot pivots off RAM onto disk; shutdown pivots back onto RAM to release the disk.

Failure Modes

• Forgetting exec before switch_root. The classic hand-rolled-initramfs bug. Without exec, PID 1 (/init) forks switch_root as a child; the real init ends up as PID 2+ and the original /init lingers as a blocked PID 1. Symptoms: systemctl complains it isn’t PID 1, or the system hangs. Fix: exec switch_root ....
• /sysroot not a mount point. If the root mount failed silently and /sysroot is an empty directory on the initramfs, switch_root fails (newroot is not the root of a mount). The real cause is upstream — the root device wasn’t found/mounted (missing storage driver, wrong root=, unfinished LUKS/RAID). Diagnose with rd.break=pre-mount / pre-pivot (see Early vs Late Userspace).
• Deleting the wrong filesystem — guarded, but instructive. The fstatfs ramfs/tmpfs check and the st_dev -xdev guard in recursiveRemove exist precisely because a naive “delete the old root” would otherwise be able to recurse into the just-mounted real root or the moved API filesystems. If you ever see “old root filesystem is not an initramfs” in early boot logs, switch_root is refusing to delete — which means the old root was not a ramfs/tmpfs (e.g. someone ran switch_root outside the boot context). That warning is the safety net working, not a bug.
• MS_MOVE of /run failing. If /run can’t be moved (e.g. it became a shared mount), switch_root force-unmounts it (MNT_FORCE), and you lose its contents — including the stash that the shutdown exitrd needs. The fix is to keep these mounts private in the initramfs.
• Confusing switch_root with pivot_root and getting EINVAL. Calling pivot_root(2) from the initramfs returns EINVAL “The current root is on the rootfs (initial ramfs) mount” (pivot_root(2)). That is the kernel telling you to use switch_root instead.

Alternatives and When to Choose Them

• pivot_root(2) / pivot_root(8) — use when the old root is a real mount you want to keep and later detach: legacy initrd, and every container runtime. It is the wrong tool from rootfs (it’s disallowed). Deep dive: pivot_root and Changing the Root.
• chroot(2) alone — changes only the apparent root directory, leaving the old root mounted and reachable. Never used as the boot pivot because it can’t reclaim the initramfs RAM and is escapable. switch_root uses chroot(".") as one step, but only after the MS_MOVE has already overmounted /.
• No pivot at all (built-in root). If the kernel has the storage driver compiled in and root= points at a simple device, you can boot with no initramfs; the kernel mounts the real root directly and runs its /sbin/init — no switch_root needed. See Mounting the Real Root Filesystem for when an initramfs is and isn’t required.

Production Notes

The two tools you actually meet in the wild are util-linux switch_root (the C tool traced above, used by systemd-based and modern initramfs) and busybox switch_root, a near-identical applet — the kernel docs note “Most other packages (such as busybox) have named this command ‘switch_root’” (ramfs-rootfs-initramfs.rst). dracut-generated initramfs images (the dominant generator on Fedora/RHEL/openSUSE) and Debian’s initramfs-tools both end their early-userspace stage with a switch_root/run-init into the real root. The single most common boot failure that drops you to a dracut:/# or (initramfs) emergency shell is the pivot not happening because the real root never mounted — the pivot itself almost never fails on its own; it is the canary for a root-discovery problem one stage earlier.

Uncertain

Verify: that systemd’s initrd-switch-root.service performs the identical delete-the-old-ramfs step as util-linux switch_root (vs. relying on the kernel to reclaim the initramfs differently). Reason: the systemd bootup.xml describes the switch-root and /sysroot handoff but does not spell out whether it recursively deletes the old rootfs contents the way switch_root.c does. To resolve: read src/core/switch-root.c in the systemd source for the exact mount/delete sequence. uncertain

See Also

• pivot_root and Changing the Root — the pivot_root(2) syscall and its container/isolation semantics; the keep-the-old-root counterpart to this note’s delete-the-old-root.
• Mounting the Real Root Filesystem — the stage before the pivot: finding and mounting the real root at /sysroot.
• The Early Userspace init Script — what the initramfs /init does before it execs switch_root.
• Early vs Late Userspace — the conceptual boundary the pivot crosses; initrd.target vs the real targets and the rd.break debug breakpoints.
• PID 1 and the init Process — why the pivot must preserve PID 1 and exec rather than fork.
• Loading the Kernel and initramfs into Memory — how the bootloader and kernel get the initramfs into RAM in the first place.
• Linux Boot and Init MOC — the parent map; this is one of the two key seams of the boot relay.