vault

BPF Bytecode and Disassembly

16 min read

BPF Bytecode and Disassembly

A compiled BPF program is just an array of 8-byte (occasionally 16-byte) instructions for the eBPF virtual machine, and learning to read it — in source, after the verifier rewrites it, and after the JIT lowers it to native code — is the single most useful debugging skill in eBPF work. The bytecode exists in three distinct forms that do not match each other: the ELF object that clang -target bpf emits (the program as compiled, before the kernel sees it), the xlated image (bpftool prog dump xlated — the eBPF as the verifier rewrote it, with helper inlining and pointer fix-ups), and the jited image (bpftool prog dump jited — the native x86-64/arm64 machine code the BPF JIT Compiler produced) (bpftool-prog.rst, v6.12). All three render in a compact textual assembly — r1 = r2, *(u64 *)(r10 - 0x8) = r1, if r0 > 0x10 goto +0x3, call bpf_map_lookup_elem#NN — produced by the kernel’s print_bpf_insn() and by libbpf’s disassembler (kernel/bpf/disasm.c, v6.12).

This note teaches how to compile, dump, and read BPF bytecode. For the instruction encoding bit-by-bit see BPF Instruction Set; for the eleven registers and stack the disassembly references see BPF Virtual Machine and Registers; for the bpftool/skeleton tooling around this see BPF Skeletons and bpftool.

Mental Model: Three Views of One Program

The most common confusion in reading BPF is treating “the bytecode” as one thing. It is three snapshots of a program at three stages of its life, and they legitimately differ:

flowchart LR
  SRC["BPF C source<br/>(restricted C)"]
  ELF["ELF object (.o)<br/>bytecode + .BTF + .BTF.ext<br/>llvm-objdump -d"]
  XLAT["xlated image<br/>(post-verifier eBPF)<br/>bpftool prog dump xlated"]
  JIT["jited image<br/>(native machine code)<br/>bpftool prog dump jited"]
  SRC -->|"clang -target bpf -g"| ELF
  ELF -->|"bpf() BPF_PROG_LOAD<br/>+ verifier rewrites"| XLAT
  XLAT -->|"JIT compiler"| JIT

The three forms of a BPF program and the tool that shows each. What it shows: llvm-objdump -d reads the ELF on disk — the eBPF exactly as the compiler emitted it, before the kernel exists in the picture; bpftool prog dump xlated reads the program out of the kernel after the verifier has rewritten it; bpftool prog dump jited reads the native code the JIT produced. The insight to take: these are not three encodings of the same instructions — the verifier changes the instructions (inlining some helper calls, adjusting offsets, inserting Spectre nospec barriers), so xlated can have more, fewer, or different instructions than the ELF, and jited is a different ISA entirely. When debugging, always ask “which view am I looking at?”

What clang -target bpf Emits

A BPF program is compiled with Clang/LLVM’s BPF backend (“The extended Berkeley Packet Filter (eBPF) backend” in the LLVM code generator, emitting file format elf64-bpf) by passing -target bpf (LLVM CodeGenerator). The output is an ordinary ELF relocatable object, but with BPF-specific sections. Compiling a minimal kprobe program with clang -O2 -g -target bpf -c prog.c -o prog.o and inspecting it with llvm-readelf -S prog.o yields sections like these (real output, LLVM 21):

[Nr] Name               Type      Flg
[ 3] kprobe/sys_execve  PROGBITS  AX    <- the program's bytecode (named by SEC())
[ 4] .relkprobe/...     REL             <- relocations for that program
[ 5] .maps              PROGBITS  WA    <- map definitions (BTF-typed)
[ 6] license            PROGBITS  WA    <- the GPL license string
[16] .BTF               PROGBITS        <- type + string info (vmlinux-style)
[17] .rel.BTF           REL
[18] .BTF.ext           PROGBITS        <- func_info + line_info + CO-RE relos
[19] .rel.BTF.ext       REL
[22] .debug_line        PROGBITS        <- DWARF line table (from -g)
[26] .symtab            SYMTAB

Several things are load-bearing here:

  • The program lives in a section named by its SEC() attribute, not in .text. A program declared SEC("kprobe/sys_execve") ends up in a PROGBITS section literally named kprobe/sys_execve. libbpf reads the section name to decide the program type and attach point. Multiple programs mean multiple such sections.
  • .maps holds the map definitions, encoded as BTF-typed zero-size structs whose member types describe the map (key type, value type, max_entries, etc.). The bytecode does not embed map file descriptors — those don’t exist until load time — so map references in the code are relocations.
  • .BTF is the BPF Type Format type/string blob (the same format as /sys/kernel/btf/vmlinux), and .BTF.ext carries func_info, line_info, and CO-RE relocation records (btf.html). .BTF.ext’s line_info is what makes source-annotated disassembly possible (covered below). These are produced from the -g debug flag; without -g you still get bytecode but no source interleaving.
  • Relocations (.relkprobe/sys_execve) patch map references and (with CO-RE) field offsets. An llvm-objdump -dr shows e.g. R_BPF_64_64 counts against the .maps section where the program loads the address of map counts.

Reading the ELF bytecode (llvm-objdump -d)

llvm-objdump -d prog.o disassembles the eBPF as compiled, needing no privilege and no running kernel — the ground truth for the compiler’s output. Real output (LLVM 21) for a program that looks up a per-key counter and increments it up to a cap of 0x10:

0000000000000000 <count_execve>:
   0:  b4 01 00 00 00 00 00 00  w1 = 0x0
   1:  63 1a fc ff 00 00 00 00  *(u32 *)(r10 - 0x4) = w1
   2:  bf a2 00 00 00 00 00 00  r2 = r10
   3:  07 02 00 00 fc ff ff ff  r2 += -0x4
   4:  18 01 00 00 00 00 00 00
      00 00 00 00 00 00 00 00  r1 = 0x0 ll
   6:  85 00 00 00 01 00 00 00  call 0x1
   7:  b4 01 00 00 01 00 00 00  w1 = 0x1
   8:  15 00 06 00 00 00 00 00  if r0 == 0x0 goto +0x6 <count_execve+0x78>
   9:  79 02 00 00 00 00 00 00  r2 = *(u64 *)(r0 + 0x0)
  10:  b4 01 00 00 00 00 00 00  w1 = 0x0
  11:  25 02 03 00 10 00 00 00  if r2 > 0x10 goto +0x3 <count_execve+0x78>
  12:  07 02 00 00 01 00 00 00  r2 += 0x1
  13:  7b 20 00 00 00 00 00 00  *(u64 *)(r0 + 0x0) = r2
  14:  b4 01 00 00 01 00 00 00  w1 = 0x1
  15:  bc 10 00 00 00 00 00 00  w0 = w1
  16:  95 00 00 00 00 00 00 00  exit

Reading it line by line teaches the whole syntax (see Textual syntax below). The left column is the instruction index, then the raw 8 (or 16) bytes, then the textual disassembly. Note instruction 4 is sixteen bytes — the wide load-immediate (lddw, opcode 0x18), here loading a map address (it disassembles as r1 = 0x0 ll, the ll suffix marking the 64-bit immediate; the 0x0 is a placeholder the relocation fills in with the real map fd at load). Note also w1/w0 (the 32-bit sub-register view of r1/r0) versus full r2, and the signed displacement in r2 += -0x4.

xlated vs jited — Dumping a Loaded Program

Once the program is loaded into the kernel (via the bpf() syscall, typically by libbpf), bpftool dumps the two in-kernel forms. The synopses, verbatim from the v6.12 man page (bpftool-prog.rst):

bpftool prog dump xlated PROG [{ file FILE | [opcodes] [linum] [visual] }]
bpftool prog dump jited  PROG [{ file FILE | [opcodes] [linum] }]

where PROG selects the program by id ID, tag TAG, name NAME, or pinned FILE.

xlated (“translated”) dumps “eBPF instructions of the programs from the kernel” — the program after the verifier has rewritten it. This is the most important and most surprising view, because it is not the same instructions the ELF held:

  • Helper-call inlining. The verifier inlines some common helpers. The classic example is bpf_map_lookup_elem on array maps: in the xlated dump the call is gone, replaced by a few inline instructions that compute the element address directly — a major performance optimisation that only shows up in xlated, never in the ELF.
  • Pointer and offset rewrites. Context-field accesses (e.g. reading a field of struct __sk_buff) are rewritten by the program type’s convert_ctx_access to the real offset in the real kernel struct; ldimm64 map-fd placeholders are filled with the actual map address; subprogram call targets are resolved.
  • Spectre hardening. The verifier may insert nospec (speculation-barrier) pseudo-instructions and bounds-clamping it deemed necessary. See BPF and Spectre Hardening.
  • Former classic programs. A converted classic BPF program shows up in xlated with the translator’s prologue (w0 = 0; w7 = 0; r6 = r1; ...) — the xlated view is where you see the cBPF→eBPF conversion result.

jited dumps the “jited image (host machine code) of the program” — native instructions. On x86-64 this is push %rbp / mov %rsp,%rbp / sub $0x...,%rsp and so on; it is no longer eBPF at all but the actual code the CPU runs. You read jited to confirm the JIT exists/works, to inspect hardening (constant blinding, retpolines), or to count real instructions for a hot path. Because it is per-architecture, jited output on arm64 looks entirely different.

The defining mental rule: ELF = what the compiler produced; xlated = what the verifier produced; jited = what the CPU runs. Disagreements between them are expected and informative, not bugs.

Textual Instruction Syntax

The eBPF assembly syntax is small and regular. It is produced by the kernel’s print_bpf_insn() in kernel/bpf/disasm.c (used by the verifier log) and, near-identically, by libbpf’s disassembler behind bpftool. The exact format strings from disasm.c (v6.12) define the grammar precisely:

  • Register move / ALU: r<dst> = r<src>, r<dst> = <imm>, r<dst> += r<src>, r<dst> &= 0xf, etc. A w prefix (w1 = 0x0) denotes the 32-bit sub-register operation (BPF_ALU class); a plain r denotes the 64-bit operation (BPF_ALU64). The kernel renders this with (%02x) %c%d %s %s%c%d where %c is 'r' or 'w'.
  • Load: r<dst> = *(<size> *)(r<src> <off>) — e.g. r2 = *(u64 *)(r0 + 0x0). <size> is u8/u16/u32/u64. The format string is (%02x) r%d = *(%s *)(r%d %+d).
  • Store from register: *(<size> *)(r<dst> <off>) = r<src> — e.g. *(u32 *)(r10 - 0x4) = w1. Format: (%02x) *(%s *)(r%d %+d) = r%d.
  • Store immediate: *(<size> *)(r<dst> <off>) = <imm>.
  • Wide load-immediate (ldimm64): r<dst> = <imm> ll (llvm-objdump) / r<dst> = <value> with a symbolic name in bpftool when it is a map fd — the only 16-byte instruction.
  • Conditional jump (reg): if r<dst> <op> r<src> goto pc<+off> — e.g. if r0 == 0x0 goto +0x6. Format: (%02x) if %c%d %s %c%d goto pc%+d. <op> is ==, !=, >, >=, <, <=, s>, s>= (signed), etc. With the JMP32 class the registers print as w<n>.
  • Conditional jump (imm): if r<dst> <op> 0x<imm> goto pc<+off> — e.g. if r2 > 0x10 goto +0x3. Format: (%02x) if %c%d %s 0x%x goto pc%+d.
  • Unconditional jump: goto pc<+off> (BPF_JA); gotol pc<+off> is the 32-bit-offset long jump; may_goto pc<+off> is the newer bounded-loop primitive.
  • Call: call <name>#<imm> for a helper ((%02x) call %s#%d) — e.g. call bpf_map_lookup_elem#1, where the #N is the helper id; call pc<+off> (call pc%s) for a BPF-to-BPF call to a subprogram; kfunc calls render with a BPF_PSEUDO_KFUNC_CALL source.
  • Exit: exit (BPF_EXIT) — return to caller with the value in r0.

The parenthesised (%02x) you see in verifier log output (e.g. (bf) r6 = r1) is the raw 1-byte opcode; it is part of the kernel’s verbose() format. In bpftool output the opcode prefix behaviour is governed by the opcodes flag (below). The signed offsets render with an explicit sign (%+d) so backward jumps show as goto -0x3.

Options: opcodes, linum, visual

The dump flags control how much detail and which annotations appear (bpftool-prog.rst, v6.12):

  • opcodes — “controls if raw opcodes should be printed as well” (xlated) / “controls if raw opcodes will be printed” (jited). For jited the man page’s own example shows it printing the full raw instruction bytes beneath each disassembled line, e.g. mov %rsp,%rbp followed by 48 89 e5 — i.e. it adds the complete machine-code bytes, not a 1-byte tag.
  • linum — “If linum is specified, the filename, line number and line column will also be displayed.” This consumes the .BTF.ext line_info records (see below) to annotate each instruction with where in the source it came from.
  • visual (xlated only) — builds a control-flow graph of the eBPF instructions and emits it in DOT format on stdout, suitable for piping into Graphviz (dot -Tpng). This is the modern echo of the 1993 paper’s CFG model: the program’s basic blocks and branch edges, visualised.
  • file FILE — write the binary image to a file instead of disassembling to stdout.
  • -p / --pretty (JSON) — bpftool is JSON-capable; with -j/-p the dump becomes structured JSON instead of text, for tooling.

Uncertain

Verify: the exact default-vs-opcodes textual format of bpftool prog dump xlated (does the default already show the (bf)-style 1-byte opcode prefix, or does opcodes add the full 8-byte encoding as it does for jited?). Reason: the v6.12 bpftool-prog.rst EXAMPLES section shows only a jited dump with opcodes; it contains no xlated example, and bpftool was not available in this environment to run (clang/llvm-objdump were, and supplied the ELF-view ground truth above, but those do not exercise the kernel-side xlated dump). To resolve: run bpftool prog dump xlated id <N> and ... opcodes on a kernel with a loaded program, or read libbpf’s dump_xlated_plain() in tools/bpf/bpftool/xlated_dumper.c. uncertain

Source-Annotated Disassembly via BTF line_info

The reason a BPF dump can interleave source code with instructions is the line_info records in .BTF.ext. Each bpf_line_info record (btf.html) ties an instruction back to a source location:

struct bpf_line_info {
    __u32 insn_off;       /* instruction this line maps to */
    __u32 file_name_off;  /* offset into BTF string table -> filename */
    __u32 line_off;       /* offset into BTF string table -> the source line text */
    __u32 line_col;       /* packed: line = line_col >> 10, column = line_col & 0x3ff */
};

In the ELF, insn_off is a byte offset from the program section start; the kernel API form uses instruction units (struct bpf_insn counts). Crucially, .BTF.ext stores not just the number but the actual source line text (line_off points at the string), so tools can print the C line without access to the original .c file. When you pass linum (or simply load a -g program and let libbpf attach the line info), the verifier log and bpftool dumps show, above an instruction, the file/line/column and the source statement — making “the verifier rejected instruction 42” actionable by pointing at the exact C line. This is also why func_info (the sibling record mapping insn_off → BTF_KIND_FUNC type) lets dumps label subprograms by name rather than pc+N.

Failure Modes and Gotchas

  • “My xlated dump doesn’t match my C / my objdump.” Expected — the verifier inlines helpers and rewrites context/map accesses. Compare source intent against xlated, not instruction-for-instruction. Use llvm-objdump -d for the un-rewritten view.
  • No source lines in the dump. You compiled without -g, so there is no .BTF.ext line_info; or the loader stripped BTF; or your bpftool/kernel lacks BTF support. Recompile with -g.
  • call -1 / unresolved helper name. The disassembler could not resolve the helper id to a name (missing BTF/kernel symbol info); the #N id is still correct. kfunc calls in particular need kernel BTF to name.
  • jited dump empty or “not jited”. The program is running on the interpreter, not JITed — the architecture lacks a JIT, or net.core.bpf_jit_enable is 0. See BPF Interpreter vs JIT.
  • Wide-immediate confusion. The 16-byte ldimm64 (r1 = ... ll) occupies two instruction slots; indexes after it jump by two. Map fds appear as this instruction with a placeholder until relocation. Miscounting it is a classic off-by-one when hand-reading offsets.
  • opcodes floods output. On a large program the raw-byte column dwarfs the disassembly; use it surgically when you suspect an encoding issue, not by default.

Alternatives and When to Choose Them

  • llvm-objdump -d prog.o — no kernel, no privilege, shows the compiler’s output. Best for: “what did clang emit?”, verifying CO-RE relocations (-dr), inspecting before loading. Cannot show verifier rewrites or the JIT.
  • bpftool prog dump xlated — needs the program loaded (and CAP_BPF/root). Best for: “what is actually running after the verifier?”, confirming helper inlining, seeing a converted classic program. The canonical debugging view.
  • bpftool prog dump jited — loaded program, root. Best for: native-code inspection, JIT/hardening confirmation, micro-optimisation.
  • The verifier log (returned by BPF_PROG_LOAD, surfaced by libbpf on failure) — the same textual syntax, but annotated with per-instruction register state and the rejection reason. The primary tool for debugging verifier errors.
  • readelf -S / llvm-readelf -S — for the ELF structure (sections, BTF presence) rather than the instructions.

Production Notes

In practice, engineers reach for bpftool prog dump xlated id <N> linum as the first move when a loaded program “isn’t doing what the source says”: the xlated view plus source lines immediately shows whether the verifier inlined or rewrote the path in question. The visual CFG dump (DOT → Graphviz) is invaluable for understanding a large program’s branch structure — for example confirming that an early-exit fast path really is reached before the expensive logic. Tools like Cilium and bpftrace ship deep BTF and rely on exactly this line_info machinery so that their generated programs remain debuggable; the move to BTF-everywhere is partly what made BPF programs introspectable in production rather than opaque blobs. When filing a kernel-BPF bug, maintainers routinely ask for the xlated dump (and the verifier log), because the ELF tells them what you meant and the xlated tells them what the kernel did.

See Also

Graph View

Backlinks