vault

libbpf and the BPF Loader

15 min read

libbpf and the BPF Loader

The [[The bpf() Syscall|bpf() syscall]] is a low, fiddly door; libbpf is the staircase everyone actually walks up. libbpf is the canonical C library that turns a Clang-compiled BPF ELF object into running, attached kernel programs: it parses the ELF, discovers the maps and programs and global variables inside it, creates the maps in the kernel, relocates the bytecode so that map references and field accesses point at the right things on this kernel, loads each program through bpf(), and attaches it to its hook. It also generates skeletons — a per-object header that gives userspace typed, string-free access to those maps and programs. Crucially, libbpf is not a third-party project bolted onto the kernel: its authoritative source lives inside the Linux tree at tools/lib/bpf and is periodically mirrored to GitHub (libbpf README), which is exactly why it tracks new kernel features first and is the de-facto standard loader that BCC, bpftool, Cilium-adjacent tooling, and the kernel’s own selftests build on.

As of mid-2026, the latest tagged libbpf release is v1.7.0 (2025-03-16), following v1.6.x; the library has had a stable 1.0 ABI since 2022 (libbpf releases). Because libbpf is userspace, its version is independent of the kernel’s — but new APIs land in tools/lib/bpf in the kernel tree first and flow out to the GitHub mirror via scripts/sync-kernel.sh.

Uncertain

Verify: that v1.7.0 (2025-03-16) is still the newest release as of the reader’s date. Reason: fetched from the GitHub releases page during this task (mid-2026); libbpf releases periodically. To resolve: check github.com/libbpf/libbpf/releases for a newer tag. #uncertain

Mental Model: A Compiler Output Needs a Linker-at-Load-Time

A normal C program is compiled to an object file, then a linker resolves symbols and a loader maps it into a process. A BPF program goes through the same shape, but the “linking and loading” must happen against the running kernel, at load time, because the kernel’s struct layouts and the set of available helpers differ between machines. libbpf is that load-time linker-loader. The compiler (clang -target bpf) emits an ELF with: BPF bytecode in code sections named by SEC(), map definitions in a .maps section, global variables in .rodata/.data/.bss, BTF type info, and relocation records marking the spots that can’t be resolved until load. libbpf reads all of that and finishes the job the compiler started.

flowchart TB
  C["prog.bpf.c"] -->|"clang -target bpf -g -O2"| OBJ["prog.bpf.o (ELF)<br/>SEC() code · .maps · .rodata/.data/.bss · BTF · relocations"]
  OBJ -->|"bpftool gen skeleton"| SKEL["prog.skel.h<br/>(embeds the .o bytes + typed structs)"]
  subgraph LIB["libbpf at runtime"]
    OPEN["__open: parse ELF, build bpf_object<br/>discover maps/progs/globals, read BTF"]
    LOAD["__load: create maps (bpf MAP_CREATE),<br/>apply CO-RE + map-fd relocations,<br/>verify+load progs (bpf PROG_LOAD)"]
    ATTACH["__attach: SEC()-driven auto-attach<br/>(kprobe, fentry, xdp, ...) -> links"]
    DESTROY["__destroy: detach, close fds, free"]
    OPEN --> LOAD --> ATTACH --> DESTROY
  end
  SKEL --> OPEN
  LOAD -->|"bpf() syscall"| K["kernel: verifier -> JIT -> maps/progs"]

The full pipeline from BPF C source to attached, running programs. What it shows: Clang produces an ELF that is incomplete on purpose — it carries relocation records instead of resolved map addresses and field offsets — and libbpf’s three runtime phases (open, load, attach) finish the work: open builds an in-memory bpf_object model from the ELF, load creates maps and rewrites the bytecode to reference them and to match this kernel’s struct layouts before submitting to the verifier, and attach wires programs to hooks. The insight to take: the heavy lifting is the load phase, where relocation happens — this is what makes one compiled .o portable across kernels, and it is the step a raw bpf() caller would have to reimplement by hand.

The bpf_object Lifecycle, Phase by Phase

libbpf models a loaded BPF application as a struct bpf_object and moves it through four phases (kernel.org libbpf overview). The phase split is deliberate: it gives userspace a window between open and load to reconfigure the object (resize maps, set global variables, disable programs) before anything is committed to the kernel.

Open (bpf_object__open / bpf_object__open_file / bpf_object__open_mem). libbpf parses the ELF and “discovers BPF maps, BPF programs, and global variables” (overview). It reads the section headers, identifies each SEC() code section as a candidate program, reads the .maps section’s BTF-defined map declarations into bpf_map structs, treats .rodata/.data/.bss as backing storage for global variables, loads the embedded BTF, and records every relocation. No syscalls happen yet — the entire object is an in-memory description. This is when you can call bpf_map__set_max_entries(), set a const volatile global the program reads as a configuration knob, or bpf_program__set_autoload(prog, false) to skip a program you don’t want on this kernel.

Load (bpf_object__load). This is the phase that does real kernel work and the heart of libbpf. It “creates BPF maps, resolves various relocations, and verifies and loads BPF programs into the kernel” (overview). In order: it issues BPF_MAP_CREATE for each map (getting back a map fd); it applies relocations to each program’s bytecode (described in detail below); it applies CO-RE relocations using the running kernel’s BTF; then for each program it issues BPF_PROG_LOAD, which runs the verifier and the JIT. If the verifier rejects a program, libbpf surfaces its log. After this phase the maps and programs exist in the kernel and are referenced by fds libbpf holds — but nothing is attached yet, so nothing runs.

Attach (bpf_object__attach_skeleton, or per-program bpf_program__attach*). libbpf “attaches BPF programs to various BPF hook points (e.g., tracepoints, kprobes, cgroup hooks)” (overview). For program types whose SEC() carries enough information to auto-attach, a single __attach() call wires them all up, returning a BPF link per program. Programs whose target can’t be inferred from the section name (e.g. a bare xdp that needs an interface index, or a perf_event program) are attached manually with the specific bpf_program__attach_xdp(prog, ifindex)-style call.

Tear down (bpf_object__destroy_skeleton / bpf_object__close). libbpf “detaches BPF programs and unloads them from the kernel. BPF maps are destroyed, and all resources used by the BPF app are freed” (overview) — concretely, it closes the link fds (detaching), then the program and map fds, dropping the kernel refcounts so the objects are reaped (unless pinned).

SEC() — From an ELF Section Name to a Program Type

A BPF program’s type is not written as an annotation in C; it is encoded in the name of the ELF section the program lands in, set by the SEC("...") macro. libbpf maps section-name prefixes to a (program_type, expected_attach_type, flags, attach_fn) tuple via a static table, section_defs[], in tools/lib/bpf/libbpf.c. Reading the actual v1.7.0 table (libbpf.c) clears up confusion that secondary summaries routinely get wrong. A few representative rows:

SEC_DEF("kprobe+",      KPROBE, 0,                SEC_NONE,       attach_kprobe),
SEC_DEF("uprobe.s+",    KPROBE, 0,                SEC_SLEEPABLE,  attach_uprobe),
SEC_DEF("tp+",          TRACEPOINT, 0,            SEC_NONE,       attach_tp),
SEC_DEF("fentry+",      TRACING, BPF_TRACE_FENTRY, SEC_ATTACH_BTF, attach_trace),
SEC_DEF("fexit.s+",     TRACING, BPF_TRACE_FEXIT, SEC_ATTACH_BTF | SEC_SLEEPABLE, attach_trace),
SEC_DEF("lsm+",         LSM,    BPF_LSM_MAC,      SEC_ATTACH_BTF, attach_lsm),
SEC_DEF("xdp",          XDP,    BPF_XDP,          SEC_ATTACHABLE_OPT),
SEC_DEF("tcx/ingress",  SCHED_CLS, BPF_TCX_INGRESS, SEC_NONE),
SEC_DEF("perf_event",   PERF_EVENT, 0,           SEC_NONE),
SEC_DEF("socket",       SOCKET_FILTER, 0,        SEC_NONE),

How to read a row: the first string is the section-name pattern; a trailing + means “this is a prefix and extra text follows” (so SEC("kprobe/do_unlinkat") matches the kprobe+ row and do_unlinkat is the function to probe). The .s infix designates a sleepable program — that’s why fexit.s+ carries the SEC_SLEEPABLE flag while plain fexit+ does not; sleepability is a property of the section spelling, not the base program type. The second/third fields are the resulting bpf_prog_type and expected_attach_type that go into the BPF_PROG_LOAD arm. The flag field controls behavior: SEC_NONE means “no auto-attach” (you must attach manually), SEC_ATTACHABLE/SEC_ATTACHABLE_OPT mark sections the skeleton’s __attach() can wire up automatically, and SEC_ATTACH_BTF means the program targets an in-kernel function identified by BTF (fentry/fexit/LSM), so libbpf must resolve the target’s attach_btf_id before loading. The last field, when present, is the C function libbpf calls to perform the attach (attach_kprobe, attach_trace, …).

This table is also why two summaries can disagree: a casual reader sees a “struct_ops” or “xdp” entry and reports “auto-attach: yes,” but whether __attach() actually fires depends on the precise flag (SEC_ATTACHABLE vs SEC_ATTACHABLE_OPT vs SEC_NONE) and on whether the program has the data it needs (an xdp program still needs an ifindex). The source table is the ground truth; the prose tables in tutorials are lossy.

The Load-Time Relocation Mechanism (the part raw bpf() can’t do for you)

The reason a BPF ELF cannot be loaded as-is — and the reason libbpf is more than a syscall wrapper — is that the compiled bytecode contains references it could not resolve at compile time. libbpf rewrites those references in bpf_object__relocate_data before each program is loaded. Reading the v1.7.0 source (libbpf.c), the two central cases are:

Map-FD relocations (RELO_LD64). When BPF C does bpf_map_lookup_elem(&my_map, &key), the compiler emits a 64-bit load-immediate (ldimm64) instruction whose immediate is a placeholder — the compiler has no idea what file descriptor my_map will get, because the map doesn’t exist until load time. After libbpf creates the map and has its fd, it patches the instruction:

case RELO_LD64:
    map = &obj->maps[relo->map_idx];
    ...
    insn[0].src_reg = BPF_PSEUDO_MAP_FD;
    insn[0].imm     = map->fd;
    break;

It writes the freshly created map’s fd into the instruction’s imm field and tags the source register as BPF_PSEUDO_MAP_FD. The verifier recognizes that pseudo-register and, during BPF_PROG_LOAD, converts the fd-bearing instruction into a real in-kernel map pointer. So the chain is: compiler emits a placeholder → libbpf creates the map and stamps its fd into the bytecode → the verifier turns the fd into a pointer. None of this is possible from a single hand-written bpf() call without replicating exactly this dance.

Global-variable relocations (RELO_DATA). Global variables don’t live in registers — libbpf backs each data section with an internal BPF map. .rodata becomes a single-entry array map that is populated with the initial values and then frozen (BPF_MAP_FREEZE) so it is read-only to userspace and the verifier can treat its contents as constants — which is how const volatile config knobs get constant-folded by the verifier. .data/.bss become writable array maps. A reference to a global is compiled as a load from a fixed offset into the section; libbpf relocates it to a BPF_PSEUDO_MAP_VALUE load against the backing map’s fd at the right offset:

case RELO_DATA:
    map = &obj->maps[relo->map_idx];
    insn[1].imm = insn[0].imm + relo->sym_off;
    ...
    insn[0].src_reg = BPF_PSEUDO_MAP_VALUE;
    insn[0].imm     = map->fd;
    break;

On top of these, libbpf applies CO-RE relocations: each __builtin_preserve_access_index field access in the source carries a relocation record describing the field by type and name, and at load time libbpf looks the field up in the running kernel’s BTF (/sys/kernel/btf/vmlinux) and rewrites the access to the correct offset for this kernel. That is the mechanism that lets one .o read task->pid on kernels where the field sits at different offsets — covered in depth in CO-RE (Compile Once Run Everywhere) and BTF and vmlinux.h. There are also RELO_EXTERN_* cases for kconfig externs, typed/typeless ksyms, and kfunc calls, each patched the same way: rewrite the instruction so the verifier can finish resolving it.

Skeletons — Typed, String-Free Access

Even with libbpf doing the loading, talking to a map by string name (bpf_object__find_map_by_name(obj, "events")) is error-prone and verbose. The skeleton removes that. Running bpftool gen skeleton prog.bpf.o > prog.skel.h produces a header that (overview) gives “direct access to all the BPF maps and BPF programs as struct fields,” eliminates string-based lookups, and embeds the BPF object bytecode directly in the header so the resulting userspace binary is self-contained — no separate .o to ship. The generated header provides the four lifecycle entry points named after the object:

  • prog__open() — allocate and open (parse the embedded ELF, build the bpf_object)
  • prog__load() — create maps, relocate, verify and load
  • prog__attach() — auto-attach the attachable programs
  • prog__destroy() — detach, close, free

A typical loader is then a dozen lines: skel = prog__open(); /* set skel->rodata->config = ...; */ prog__load(skel); prog__attach(skel); then read events off skel->maps.events. The skeleton fields are typed — skel->bss->counter is a real __u64, mmap-backed and shared with the kernel-side program, so userspace and kernel see the same memory without any syscall per access (because .data/.bss/.rodata maps are created BPF_F_MMAPABLE and mmap’d into the process).

A second, more specialized variant is the light skeleton (bpftool gen skeleton -L). Instead of embedding the full libbpf-dependent loader, it embeds a tiny self-contained loader and a loader BPF program that performs map creation and program loading from inside the kernel via the in-kernel bpf_sys_bpf/kern_sys_bpf path (the whitelisted re-entry into __sys_bpf noted in The bpf() Syscall). Because it doesn’t link against full libbpf and can run very early, the light skeleton is what lets BPF be loaded during kernel boot and from kernel modules (syscall.c comment at v6.12). For ordinary userspace tools, the regular skeleton is the right choice; the light skeleton is a niche for very-early or self-contained loading. See BPF Skeletons and bpftool for the generator and the field layout in depth.

Failure Modes and Gotchas

  • Verifier rejection at load. A BPF_PROG_LOAD failure surfaces through libbpf as a load error with the verifier log attached; libbpf automatically retries with a larger log buffer if the first was truncated. The fix lives in the program, not the loader — see Reading and Debugging Verifier Errors.
  • Missing kernel BTF for CO-RE. CO-RE relocations need /sys/kernel/btf/vmlinux, which requires the kernel to be built with CONFIG_DEBUG_INFO_BTF=y. On a kernel without it, CO-RE field relocations cannot be resolved and load fails; the workaround is shipping an external BTF blob for the target kernel.
  • SEC() typos and stale names. A misspelled or renamed section (e.g. legacy SEC("classifier") vs modern SEC("tcx/ingress")) either fails to match any section_defs[] row (load error) or matches a deprecated row. libbpf 1.0 made section-name matching strict — the days of fuzzy prefix matching are over (Nakryiko, libbpf 1.0).
  • Objects vanish when the loader exits. This is a property of the fd object model, not a libbpf bug: when your process ends, libbpf’s fds close and unpinned objects are reaped. Long-lived programs must be pinned or attached via a link that holds the reference.
  • Global init only takes before load. Setting skel->rodata->cfg must happen between __open and __load — .rodata is frozen at load, so writes after load are rejected.

Alternatives and When to Choose Them

libbpf is the default, but it is not the only loader, and the alternatives illuminate what it does:

  • BCC (BPF Compiler Collection). BCC’s model is runtime compilation: it embeds BPF C as a string and compiles it on the target machine using an embedded Clang/LLVM, which means every host running a BCC tool needs a full LLVM toolchain and matching kernel headers present (BCC INSTALL.md). That is heavy (hundreds of megabytes of LLVM) and fragile (compilation can fail on a host you didn’t test). libbpf + CO-RE replaced this for most tools: compile once on the developer’s machine, ship a small static binary, relocate at load. Modern BCC even offers a libbpf-based path for exactly this reason. Reach for BCC’s runtime-compile model only when you genuinely need to generate program source dynamically at runtime.
  • cilium/ebpf (Go). A pure-Go reimplementation of the ELF loader and CO-RE relocation that is “compatible with the upstream libbpf” but does not wrap it via cgo (ebpf-go loader docs). It parses the ELF into Go CollectionSpec/ProgramSpec/MapSpec values, lets you modify them in Go, then loads to the kernel; its bpf2go tool plays the role of skeleton generation. Choose it when your control plane is Go and you want to avoid cgo (a major reason Cilium’s userspace is Go).
  • Raw bpf() / aya (Rust) / other bindings. Hand-rolled bpf() is for selftests and the truly minimal. aya is a Rust ecosystem that, like cilium/ebpf, reimplements loading without libC dependency. The common thread: any serious loader must reimplement ELF parsing, map creation, relocation, and CO-RE — which is precisely the body of work libbpf standardized first.

Production Notes

libbpf’s status as the de-facto standard is structural, not just popular: because its source of truth is tools/lib/bpf in the kernel tree and it is mirrored out, new kernel features (a new map type, a new attach mode, a new SEC() spelling) appear in libbpf essentially as they merge, and the kernel’s own selftests exercise it continuously. This is why distros ship libbpf as a system library and why bpftool (also in tools/bpf) and most observability agents build on it. The library’s 1.0 release (2022) froze a clean, stable ABI and removed a decade of deprecated APIs; subsequent 1.x releases (through v1.7.0, 2025-03-16) add features — exclusive map creation, kprobe-session programs, module-qualified fentry targets, indirect-jump-table support for newer BPF ISA — without ABI breaks (releases, v1.5.0 notes). For a team building eBPF tooling in C, the practical recipe is: write *.bpf.c, compile with clang -target bpf -g -O2, generate vmlinux.h from the kernel’s BTF, bpftool gen skeleton, link against libbpf, ship one static binary — the workflow that made eBPF deployable at industry scale.

See Also

Graph View

Backlinks