vault

Map Pinning and bpffs

17 min read

Map Pinning and bpffs

A BPF map is a kernel object referenced by a file descriptor (fd); when the last fd referring to it closes, the kernel can free it. Pinning breaks that coupling: it creates a named entry in a special filesystem — the BPF filesystem (bpffs), conventionally mounted at /sys/fs/bpf — that itself holds a reference to the map. As long as that path exists, the map stays alive even after the process that created it has exited, and any other process can re-open the same map by path. Pinning is implemented by the BPF_OBJ_PIN command of the bpf() syscall, which routes into bpf_obj_pin_user() in kernel/bpf/inode.c; re-opening is BPF_OBJ_GET → bpf_obj_get_user() (per inode.c at v6.12). This note owns the filesystem mechanics: what bpffs is, how a pin entry is made, and how libbpf automates it. The general reference-count and lifetime model across all BPF object types lives in its sibling BPF Object Pinning and Lifetime.

Mental Model

Think of a BPF map the way you think of an anonymous pipe or an unnamed temp file: it exists only as long as someone holds a handle to it. The handle is a file descriptor. When your loader program exits, the kernel closes its fds, the map’s reference count drops to zero, and the map is gone. That is fine for a self-contained tool that loads, attaches, runs, and tears everything down on exit — but it is fatal for the common pattern where one process loads and configures a map and a different process (or a later invocation of the same process) wants to read or update it.

Pinning is the BPF answer to “give this object a name on disk so it outlives me.” The name lives in bpffs, a tiny in-kernel pseudo-filesystem whose only job is to hold BPF objects. When you pin a map at /sys/fs/bpf/my_map, the kernel creates an inode at that path whose i_private field points at your struct bpf_map, and — crucially — bumps the map’s reference count so the filesystem entry counts as a holder. The map now has two reasons to stay alive: your fd, and the pin. Close your fd (or exit) and the pin alone keeps it. Another process calls BPF_OBJ_GET on the same path, the kernel hands back a fresh fd referring to the same underlying map, and the two processes now share state.

flowchart LR
  subgraph P1["Process A (loader)"]
    FD1["fd 5 -> map"]
  end
  subgraph KERN["Kernel"]
    MAP["struct bpf_map<br/>refcnt / usercnt"]
    subgraph BPFFS["bpffs @ /sys/fs/bpf"]
      PIN["inode 'my_map'<br/>i_private -> map<br/>(holds 1 ref)"]
    end
  end
  subgraph P2["Process B (reader, later)"]
    FD2["fd 3 -> map"]
  end
  FD1 -->|BPF_OBJ_PIN| PIN
  FD1 --> MAP
  PIN -->|reference| MAP
  FD2 -->|BPF_OBJ_GET| PIN
  FD2 --> MAP

How a pin decouples a map’s lifetime from any single process. What it shows: Process A creates the map (held by fd 5) and pins it into bpffs; the pin’s inode holds its own reference to the same struct bpf_map. Even after A exits and fd 5 closes, the pin keeps the map’s reference count above zero. Process B, started later, calls BPF_OBJ_GET on the path and receives a brand-new fd (fd 3) to the very same map. The insight to take: the pin is a third kind of reference holder alongside fds and attachments — it turns an ephemeral, fd-scoped object into a named, shareable, persistent one without changing the map itself.

What bpffs Actually Is

bpffs is a real, registered Linux filesystem type — its source is kernel/bpf/inode.c, whose opening comment describes it as a “Minimal file system backend for holding eBPF maps and programs, used by bpf(2) object pinning” (inode.c v6.12, authored by Daniel Borkmann). It is registered as struct file_system_type bpf_fs_type with name "bpf" and the flag FS_USERNS_MOUNT, and its superblock carries the magic number BPF_FS_MAGIC (0xcafe4a11). Unlike ext4 or xfs, it is not backed by a block device — it is a purely in-memory pseudo-filesystem, in the same family as tmpfs, proc, and sysfs. Nothing it contains is written to disk; a bpffs mount is just a namespace of directories and special inodes that point at live kernel objects.

A bpffs mount is created with an ordinary mount(2):

mount -t bpf bpffs /sys/fs/bpf

On most modern distributions systemd mounts a bpffs instance at /sys/fs/bpf automatically during early boot, which is why that path is the universal convention (the ebpf.io docs note that if your distribution does not mount it automatically, “you can do so manually by executing mount -t bpf bpffs /sys/fs/bpf as root”, per docs.ebpf.io). You can mount additional instances anywhere; each mount is an independent tree. In the kernel, bpf_fill_super() builds the superblock, sets sb->s_op = &bpf_super_ops, marks the root directory with the inode operations bpf_dir_iops, and sets the sticky bit S_ISVTX on the root so that — like /tmp — users can only remove their own entries (inode.c v6.12, bpf_fill_super). Mounting a bpffs from outside the initial user namespace requires CAP_SYS_ADMIN (bpf_fill_super rejects fc->user_ns != &init_user_ns && !capable(CAP_SYS_ADMIN)).

Inside a bpffs you can create directories with mkdir(2) — bpf_dir_iops wires up .mkdir, .rmdir, .rename, .unlink, and even .symlink — so pin trees can be organized hierarchically (for example, one subdirectory per application). The three kinds of leaf entries a bpffs can hold are programs, maps, and links, distinguished internally by enum bpf_type (BPF_TYPE_PROG, BPF_TYPE_MAP, BPF_TYPE_LINK). Each leaf is a special inode whose i_private pointer is the kernel object; the file contents are not a byte stream you can read() arbitrarily — for a map, cat-ing the pin shows a small fdinfo-style summary rather than the map’s data (the kernel comment notes bpffs_map_fops exists only so “cat bpffs/pathto/a-pinned-map” prints metadata, not key/value contents).

One pin entry, one filesystem reference. The pin is a hard reference on the object, not a weak pointer or a symlink to an fd. This is exactly why removing the pin (an unlink/rm) is what eventually lets the object die — covered under BPF Object Pinning and Lifetime.

Mechanical Walk-through: BPF_OBJ_PIN

When userspace pins a map, it fills a union bpf_attr with the map’s fd in attr.bpf_fd and the target path in attr.pathname, then issues bpf(BPF_OBJ_PIN, &attr, size). The dispatch in kernel/bpf/syscall.c is bpf_obj_pin():

static int bpf_obj_pin(const union bpf_attr *attr)
{
	int path_fd;
 
	if (CHECK_ATTR(BPF_OBJ) || attr->file_flags & ~BPF_F_PATH_FD)
		return -EINVAL;
 
	/* path_fd has to be accompanied by BPF_F_PATH_FD flag */
	if (!(attr->file_flags & BPF_F_PATH_FD) && attr->path_fd)
		return -EINVAL;
 
	path_fd = attr->file_flags & BPF_F_PATH_FD ? attr->path_fd : AT_FDCWD;
	return bpf_obj_pin_user(attr->bpf_fd, path_fd,
				u64_to_user_ptr(attr->pathname));
}

Line by line: CHECK_ATTR(BPF_OBJ) rejects the call if any field past the last valid one (path_fd) is non-zero, a forward-compat guard. The only accepted flag is BPF_F_PATH_FD; if set, attr.path_fd is treated as a directory fd that pathname is resolved relative to (the *at()-style dirfd + pathname convention, added by commit cb8edce to avoid symlink races and to support detached bpffs mounts). If the flag is absent, path_fd defaults to AT_FDCWD so pathname is an ordinary absolute or cwd-relative path.

The real work is in bpf_obj_pin_user() → bpf_obj_do_pin() in inode.c. First, bpf_fd_probe_obj(ufd, &type) looks up the object behind the fd, trying map, then program, then link, and records which bpf_type it is. Then:

static int bpf_obj_do_pin(int path_fd, const char __user *pathname, void *raw,
			  enum bpf_type type)
{
	...
	dentry = user_path_create(path_fd, pathname, &path, 0);
	...
	dir = d_inode(path.dentry);
	if (dir->i_op != &bpf_dir_iops) {
		ret = -EPERM;           /* parent must be inside a bpffs */
		goto out;
	}
	mode = S_IFREG | ((S_IRUSR | S_IWUSR) & ~current_umask());
	ret = security_path_mknod(&path, dentry, mode, 0);
	...
	switch (type) {
	case BPF_TYPE_PROG: ret = vfs_mkobj(dentry, mode, bpf_mkprog, raw); break;
	case BPF_TYPE_MAP:  ret = vfs_mkobj(dentry, mode, bpf_mkmap,  raw); break;
	case BPF_TYPE_LINK: ret = vfs_mkobj(dentry, mode, bpf_mklink, raw); break;
	...
	}
}

Three things matter here. (1) user_path_create() resolves the parent directory and prepares to create a new dentry at the final path component — if the path already exists, the pin fails with -EEXIST. (2) The check dir->i_op != &bpf_dir_iops enforces that the target’s parent directory is inside a bpffs — you cannot pin a map onto ext4 or /tmp; the kernel returns -EPERM if you try. This is why a bpffs must be mounted before pinning. (3) For a map, vfs_mkobj(dentry, mode, bpf_mkmap, raw) creates the inode; bpf_mkmap() → bpf_mkobj_ops() sets the inode’s operations to bpf_map_iops, stashes the struct bpf_map * in inode->i_private, and — back in the bpf_obj_pin_user path — the object’s reference count was already bumped by bpf_fd_probe_obj (via bpf_map_get_with_uref). On any failure after the probe, bpf_obj_pin_user calls bpf_any_put(raw, type) to drop that bump; on success, the reference is transferred to the inode and released only when the inode is freed. The detailed semantics of that reference (a map bumps both its counters) belong to BPF Object Pinning and Lifetime.

Mechanical Walk-through: BPF_OBJ_GET

Re-opening a pinned map is the mirror image. Userspace sets attr.pathname to the pin path (and leaves attr.bpf_fd == 0) and issues bpf(BPF_OBJ_GET, &attr, size). The kernel runs bpf_obj_get_user() → bpf_obj_do_get():

static void *bpf_obj_do_get(int path_fd, const char __user *pathname,
			    enum bpf_type *type, int flags)
{
	...
	ret = user_path_at(path_fd, pathname, LOOKUP_FOLLOW, &path);
	...
	inode = d_backing_inode(path.dentry);
	ret = path_permission(&path, ACC_MODE(flags));   /* check r/w perms on the pin */
	...
	ret = bpf_inode_type(inode, type);               /* prog? map? link? */
	...
	raw = bpf_any_get(inode->i_private, *type);      /* take a NEW reference */
	...
}

It resolves the path, checks that the caller has the requested access mode against the pin file’s permission bits (so file permissions on the pin govern who may re-open it), determines the object type from the inode’s operations, and then calls bpf_any_get(inode->i_private, type) — which for a map is bpf_map_inc_with_uref() — to take a fresh reference to the same underlying object. Back in bpf_obj_get_user(), that object is wrapped in a new fd with bpf_map_new_fd(raw, f_flags) and returned to userspace. The flags (BPF_F_RDONLY/BPF_F_WRONLY, validated by bpf_get_file_flag) let a process open a pinned map read-only even if it was created read-write. The new fd is a completely independent handle: the original creator could have exited long ago; what the new opener gets is a reference to the live object that the pin has been keeping alive.

The net effect: BPF_OBJ_PIN adds a reference and a name; BPF_OBJ_GET follows the name to acquire another reference. Neither copies the map — both processes operate on one shared struct bpf_map.

Pinning From the Command Line and From libbpf

The most direct way to pin from a shell is bpftool. To pin an existing map by id:

# list maps, find the id
bpftool map show
# pin map id 42 at /sys/fs/bpf/counters
bpftool map pin id 42 /sys/fs/bpf/counters
# later, dump the pinned map by its path
bpftool map dump pinned /sys/fs/bpf/counters

bpftool map pin opens the map by id (taking an fd), then issues BPF_OBJ_PIN for that fd against the path — exactly the syscall path above.

In a C loader, the libbpf wrappers are bpf_map__pin() and bpf_map__unpin(). From the libbpf API docs: bpf_map__pin() “creates a file that serves as a ‘pin’ for the BPF map. This increments the reference count on the BPF map which will keep the BPF map loaded even after the userspace process which loaded it has exited” (per libbpf API reference). A minimal explicit pin:

struct my_skel *skel = my_skel__open_and_load();
/* pin the "events" map by hand */
err = bpf_map__pin(skel->maps.events, "/sys/fs/bpf/events");
if (err) { /* -EEXIST if already pinned, -EPERM if not in a bpffs, ... */ }

Auto-pinning: LIBBPF_PIN_BY_NAME

libbpf can pin maps automatically at load time, driven by a per-map attribute baked into the BPF object’s BTF. In the BPF C source you annotate a map’s definition with __uint(pinning, LIBBPF_PIN_BY_NAME):

struct {
	__uint(type, BPF_MAP_TYPE_HASH);
	__uint(max_entries, 1024);
	__type(key, __u32);
	__type(value, __u64);
	__uint(pinning, LIBBPF_PIN_BY_NAME);   /* auto-pin this map */
} events SEC(".maps");

LIBBPF_PIN_BY_NAME is one of two values of enum libbpf_pin_type defined in libbpf’s bpf_helpers.h: LIBBPF_PIN_NONE (the default, no pinning) and LIBBPF_PIN_BY_NAME whose comment reads “pin maps by name (in /sys/fs/bpf by default)” (libbpf bpf_helpers.h v1.4.5). At load time libbpf computes a pin path of the form <pin_root_path>/<map_name> — the root defaults to /sys/fs/bpf, overridable via the pin_root_path field of bpf_object_open_opts, documented in libbpf.h as: “maps that set the ‘pinning’ attribute in their definition will have their pin_path attribute set to a file in this directory, and be auto-pinned to that path on load; defaults to /sys/fs/bpf” (libbpf libbpf.h v1.4.5).

The auto-pinning feature has a second, load-bearing behavior beyond persistence: map reuse. If a pin already exists at the computed path when bpf_object__load() runs, libbpf does not create a fresh map — it opens the existing pinned map, verifies that its attributes (type, key/value size, max entries, flags) match the definition, and reuses it; only if no pin exists does it create a new map and pin it there. This was the explicit motivation when Toke Høiland-Jørgensen added the feature in late 2019: it lets independent BPF applications share a map by agreeing on a pin path, and lets a tool transparently rediscover its own map across restarts (netdev patch v5, 2019). You can override the computed path per map with bpf_map__set_pin_path() before bpf_object__load(), including setting nothing to fall back to auto, or disable it for one map.

”Load once, attach later” / detached BPF

Pinning is the foundation of the decoupled deployment pattern: one privileged step loads programs and maps and pins everything into bpffs; later, unprivileged or separate steps re-open the pins by path to read maps, attach programs, or hand fds to other tools. bpftool prog loadall ... pinmaps <dir> does exactly this — loads an object and pins each map under a directory — so that a downstream process can bpf_obj_get() each map without ever touching the original loader. Because the pin holds the reference, the loader can exit immediately; the BPF subsystem persists entirely in bpffs. This is how systems like Cilium keep their datapath maps (policy, conntrack, LB state) alive across agent restarts: the maps are pinned, the agent re-attaches to them on startup instead of recreating them, preserving in-flight state.

Failure Modes

  • -EINVAL “invalid argument” on pin. Usually a stale or malformed bpf_attr — for example setting path_fd without the BPF_F_PATH_FD flag, which the kernel explicitly rejects (if (!(attr->file_flags & BPF_F_PATH_FD) && attr->path_fd) return -EINVAL;).
  • -EPERM “operation not permitted” on pin. The target’s parent directory is not inside a bpffs. The classic case is pinning to a path under a directory where no bpffs is mounted (e.g. /tmp/foo or a /sys/fs/bpf that was never mounted). Verify with mount | grep bpf and mount -t bpf bpffs /sys/fs/bpf if missing.
  • -EEXIST “file exists” on pin. A pin already lives at that path. BPF_OBJ_PIN never overwrites — user_path_create fails if the final component exists. Unpin (rm/bpftool ... unpin) first, or choose a new path.
  • -ENOENT on get. No pin at the path (typo, or the pin was removed). Note that bpf_obj_get follows symlinks (LOOKUP_FOLLOW), so a dangling symlink also yields -ENOENT.
  • -EACCES / -EPERM on get. The caller lacks the requested access mode on the pin file (its rw permission bits) — bpf_obj_do_get runs path_permission(&path, ACC_MODE(flags)). Tighten or loosen the pin’s file mode, or open with BPF_F_RDONLY.
  • “Reused map has wrong attributes.” With auto-pinning, if an existing pin’s map does not match the new definition (different max_entries, value size, etc.), bpf_object__load() fails rather than silently using a mismatched map. Either delete the stale pin or reconcile the definition. This is a real footgun when a map’s shape changes between releases of a tool that auto-pins.
  • Map “leaks” / never frees. A pin is a reference. If you pin a map and forget to rm the path, the map stays resident forever, surviving every process. Persistent pins are intentional; orphaned ones are a slow memory leak. bpftool map show plus walking /sys/fs/bpf finds them.

Alternatives and When to Choose Them

The alternative to pinning a map is not pinning it and keeping its fd alive some other way. Options: (1) Keep the loader running and never pin — simplest, but the map dies with the process, so it suits one-shot tools and long-lived agents that own their maps. (2) Pass the fd over a UNIX domain socket with SCM_RIGHTS, or inherit it across fork() — both share a map between cooperating processes without any filesystem entry (the bpf(2) man page documents both, noting “file descriptors referring to eBPF objects can be transferred over UNIX domain sockets”). This avoids bpffs but requires the processes to coordinate at runtime and gives no persistence past the last fd. (3) Pin — the only option that gives a stable, name-based handle and survives the loader exiting and needs no live coordination. Choose pinning whenever the map must outlive its creator, be discoverable by path, or be shared by processes that are not related by fork and not co-running. For a transient, single-process tool, skip it.

For pinning programs and links rather than maps, the same BPF_OBJ_PIN/BPF_OBJ_GET machinery applies (and libbpf’s bpf_program__pin/bpf_link__pin mirror bpf_map__pin); the cross-cutting reference-and-lifetime model for all three is BPF Object Pinning and Lifetime. For the attachment-lifetime angle specifically — pinning a link to keep an attachment alive — see BPF Links and Attachment Lifecycle.

Production Notes

In production, /sys/fs/bpf is treated as a shared namespace and pins are organized by application: Cilium pins its maps under /sys/fs/bpf/tc/globals/, and recovers them on agent restart so connection-tracking and load-balancer state survives upgrades. Systemd units that ship BPF often declare a private bpffs sub-mount or rely on the global one and use RuntimeDirectory-style cleanup to remove stale pins. A standard operational hazard is the stale-pin upgrade: a tool changes a map’s definition between versions but the old pin still exists, so the new version either reuses an incompatible map (if it does not validate) or fails to load (libbpf validates, which is safer but louder) — upgrade procedures must explicitly unpin obsolete maps. Because a bpffs mount is per-mount-namespace, containers that want to share pinned maps with the host must bind-mount /sys/fs/bpf in, or use the BPF_F_PATH_FD dirfd form against a passed-in mount fd. Note also that the delegate_cmds/delegate_maps/delegate_progs/delegate_attachs mount options on a bpffs are not about pinning at all — they configure BPF token delegation (§9); they share the filesystem but are a separate concern.

Uncertain

Verify: the exact release that first shipped BPF_F_PATH_FD / the path_fd field for BPF_OBJ_PIN/BPF_OBJ_GET. Reason: the introducing commit (cb8edce) was committed 2023-05-23 by Daniel Borkmann via the bpf-next tree, which lands in the next merge window after 6.4 (so most likely 6.5, released 2023-08-27), but I could not confirm the exact tag by querying the kernel git directly. The load-bearing fact is solid: the field and flag are present in the v6.12 source I read directly, so both 6.12 LTS and 6.18 LTS support dirfd-relative pinning. To resolve: git tag --contains cb8edce28073a906401c9e421eca7c99f3396da1 against torvalds/linux. uncertain

See Also

Graph View

Backlinks