Instruction Dispatch and Computed Gotos

After the evaluation loop executes one bytecode instruction, it must dispatch — jump to the machine code that handles the next opcode. How that jump is structured turns out to dominate interpreter performance, because for a fast bytecode the dispatch itself can cost more than the work the opcode does. The textbook approach is one big switch (opcode) statement. CPython instead prefers, where the compiler supports it, computed gotos (also called threaded code): a table of label addresses, opcode_targets[], indexed by the opcode, with a goto *opcode_targets[opcode] replicated at the end of every opcode handler. The win is not fewer instructions — it is better CPU branch prediction: each handler gets its own indirect-branch site with its own history, so the predictor can learn the correlations between successive opcodes instead of mispredicting a single shared switch jump. The original change reported the giant switch could be made on the order of 15–20% faster this way (bpo-4753 / gh-49003) — though, as discussed below, that magnitude is dated. Python 3.14 adds a third option, the tail-call interpreter, which reaches the same goal by a different route — one C function per opcode, chained by guaranteed tail calls (What’s New in 3.14).

This note is about the dispatch mechanism only — the jump from one opcode to the next. The structure of the instructions being dispatched is Python Bytecode Instruction Set; the loop that holds the dispatch is The CPython Evaluation Loop; the runtime rewriting of opcodes is The Specializing Adaptive Interpreter (one sentence appears below, no more).

Mental Model

Picture the eval loop as a trampoline. The body of each opcode handler does its work — pop operands, compute, push result — and then, instead of falling out to a central decision point, it bounces straight to the next handler. The dispatch question is: how is that bounce implemented?

flowchart TD
    subgraph switch["switch dispatch (portable)"]
        SW["one shared\nindirect jump\n(the switch)"]
        SW --> H1["op A body"] --> SW
        SW --> H2["op B body"] --> SW
        SW --> H3["op C body"] --> SW
    end
    subgraph goto["computed-goto dispatch (threaded)"]
        G1["op A body\ngoto *targets[next]"] --> G2["op B body\ngoto *targets[next]"]
        G2 --> G3["op C body\ngoto *targets[next]"]
        G3 -.-> G1
    end

Figure: the two classic dispatch shapes. Left — a switch funnels every opcode through one indirect jump, so the CPU’s branch-target predictor sees a single site that hops everywhere and mispredicts constantly. Right — computed gotos replicate the jump at the tail of each handler, so each site has its own prediction history and can learn that, e.g., a comparison is almost always followed by a conditional jump. The insight: the two emit nearly the same total instructions; the difference is entirely in how many indirect-branch mispredictions the CPU suffers.

Why Dispatch Cost Dominates

A bytecode interpreter spends its life in a tight loop: fetch the next instruction, decode its opcode, jump to the handler, run a handful of operations, repeat. For a heavyweight opcode like CALL, the handler does enough that the dispatch overhead is noise. But Python programs are dense in cheap opcodes — LOAD_FAST, STORE_FAST, COMPARE_OP, POP_JUMP_IF_FALSE — whose handlers are only a few instructions. For those, the cost of getting to the handler (the indirect jump, and especially a mispredicted indirect jump that flushes the CPU pipeline) is a large fraction of the total. Modern out-of-order CPUs hide a correctly-predicted branch almost for free but pay 15–20+ cycles for a misprediction. So the dispatch strategy is, in effect, a branch-prediction strategy. This is exactly why the choice is worth a whole note.

Switch Dispatch — the Portable Baseline

The simplest correct dispatcher is a C switch on the opcode, with one case per instruction, wrapped in a for(;;) loop. The internal documentation describes the conceptual structure this way: “the interpreter consists of a loop that iterates over the bytecode instructions, executing each of them via a switch statement that has a case implementing each opcode” (InternalDocs/interpreter.md). The actual cases are generated from Python/bytecodes.c (see Python Bytecode Instruction Set), not hand-written.

This works everywhere and is what CPython falls back to on any compiler lacking the computed-goto extension. It has two costs. First, the C99 standard requires a switch to behave as if it checks the value against the case range — a bounds check the compiler is generally obliged to keep. Second, and far more important, a switch compiles to a single indirect jump (through a jump table) shared by every iteration. From the CPU branch predictor’s standpoint there is one indirect-branch instruction in the whole interpreter, and it targets a different handler on essentially every execution. The predictor, which keys on the address of the branch site, has no way to specialize: it sees one chaotic site and mispredicts a large fraction of dispatches.

Computed Gotos — Threaded Code

The fix is to give each opcode handler its own dispatch site. CPython does this with the GCC/Clang labels-as-values extension (also in modern Clang): the &&label operator yields the address of a C label as a void*, and goto *expr jumps to a computed address (GCC labels-as-values). This is the classic threaded-code technique.

CPython builds a static table mapping each opcode to its handler’s label address. From Python/opcode_targets.h (which is #included into the eval-loop function so the labels are in scope):

static void *opcode_targets[256] = {
    &&TARGET_CACHE,
    &&TARGET_BINARY_SLICE,
    &&TARGET_BUILD_TEMPLATE,
    &&TARGET_BINARY_OP_INPLACE_ADD_UNICODE,
    ...
};

Every entry is the address (&&TARGET_…) of a label inside the loop; unused opcode slots point at a shared &&_unknown_opcode handler so the 256-entry table is total. The macros that wire this up live in ceval_macros.h. The opcode is decoded by NEXTOPARG() (covered in Python Bytecode Instruction Set), then dispatch goes through DISPATCH() → DISPATCH_GOTO(). The three-way definition of DISPATCH_GOTO is the heart of the whole story:

#if Py_TAIL_CALL_INTERP
#   define DISPATCH_GOTO() \
        do { Py_MUSTTAIL return (INSTRUCTION_TABLE[opcode])(TAIL_CALL_ARGS); } while (0)
#elif USE_COMPUTED_GOTOS
#  define DISPATCH_GOTO() goto *opcode_targets[opcode]
#else
#  define DISPATCH_GOTO() goto dispatch_opcode
#endif

Read the middle branch: when USE_COMPUTED_GOTOS is set, dispatch is literally goto *opcode_targets[opcode] — index the table by opcode, jump to that address. The else branch (goto dispatch_opcode) is the switch fallback, where dispatch_opcode is a label sitting on the switch statement. The first branch is the tail-call interpreter, discussed below.

The matching TARGET macro defines what a “case” is in each mode:

#if Py_TAIL_CALL_INTERP
#   define TARGET(op) Py_PRESERVE_NONE_CC PyObject *_TAIL_CALL_##op(TAIL_CALL_PARAMS)
#elif USE_COMPUTED_GOTOS
#  define TARGET(op) TARGET_##op:
#else
#  define TARGET(op) case op: TARGET_##op:
#endif

Under computed gotos, TARGET(LOAD_FAST) expands to a bare label TARGET_LOAD_FAST: (the target of the table entry). Under switch, it expands to case LOAD_FAST: TARGET_LOAD_FAST: — both a case and a label, so the same generated handler body serves either mode. The full DISPATCH() macro ties decode and jump together:

#define DISPATCH() \
    { \
        assert(frame->stackpointer == NULL); \
        NEXTOPARG(); \
        PRE_DISPATCH_GOTO(); \
        DISPATCH_GOTO(); \
    }

Crucially, DISPATCH() is invoked at the end of every opcode handler, not once at the top of the loop. That replication is the entire point: it places a distinct goto *opcode_targets[opcode] instruction at the tail of each handler. Each of those instructions is a separate indirect-branch site with its own entry in the CPU’s branch-target buffer (BTB).

Why per-site history actually helps

Replicating the jump would be pointless if successive opcodes were random. They are not. Bytecode streams are highly correlated: a COMPARE_OP is almost always followed by a POP_JUMP_IF_FALSE; a LOAD_FAST is very often followed by another LOAD_FAST; a FOR_ITER body has a stable shape. With one shared switch site, the predictor sees the union of all those successors and cannot distinguish them. With a per-handler site, the COMPARE_OP handler’s dispatch instruction is, in this program, almost always jumping to POP_JUMP_IF_FALSE — a near-constant target the BTB predicts correctly. The predictor learns the transition, not just the destination. That correlation is what converts “one chaotic branch” into “many well-behaved branches.”

The configuration plumbing auto-enables the optimization where available, from ceval_macros.h:

#ifdef HAVE_COMPUTED_GOTOS
    #ifndef USE_COMPUTED_GOTOS
    #define USE_COMPUTED_GOTOS 1
    #endif
#else
    #if defined(USE_COMPUTED_GOTOS) && USE_COMPUTED_GOTOS
    #error "Computed gotos are not supported on this compiler."
    #endif
    #undef USE_COMPUTED_GOTOS
    #define USE_COMPUTED_GOTOS 0
#endif

configure probes the compiler for the labels-as-values extension and #define HAVE_COMPUTED_GOTOS; if present and the builder did not force it off, USE_COMPUTED_GOTOS defaults to 1. A builder can override with -DUSE_COMPUTED_GOTOS=0 (to benchmark the switch baseline) or =1 (which errors out if the compiler can’t do it).

The Historical Numbers and a Subtlety

Computed gotos were proposed for CPython by Antoine Pitrou in 2008 (bpo-4753 / gh-49003, “Faster opcode dispatch on gcc”) and enabled by default on supporting compilers shortly after (bpo-9203 / gh-53449, “Use computed gotos by default”). The contemporaneous reports measured the change with pybench across several CPUs (the issue carries benchmark attachments for Athlon64, Core 2, and PowerPC G5), and the widely-cited summary of that era’s result is that the enormous switch could be made on the order of 15–20% faster, with the gain attributed to avoiding the C99 switch bounds check and — more importantly — better CPU branch prediction reducing pipeline flushes.

Uncertain (as of 2026-06-01)

Verify: the precise “15–20%” magnitude and its durability on current hardware/compilers. Reason: the figure is from the 2008–2010 bpo-4753 measurements on that era’s CPUs and GCC; modern branch predictors (TAGE-class) and modern compilers behave very differently, and independent 2025 analysis put the benefit at roughly 2–4% on modern processors — with a switch interpreter under recent Clang sometimes matching it via auto-optimization (Nelson Elhage 2025). The direction (computed gotos historically faster) is well established and not in doubt; only the magnitude is dated and hardware-dependent. To resolve: re-benchmark a 3.14.x build with USE_COMPUTED_GOTOS=0 vs =1 on current hardware. uncertain

A modern subtlety complicates the picture: compilers sometimes defeat the optimization by merging the duplicated DISPATCH jumps back into a single shared site (tail-merging / cross-jumping), collapsing computed gotos into the equivalent of a switch. The rationale and the failure are stated precisely in CPython issue gh-129987, “computed-goto interpreter: Prevent the compiler from merging DISPATCH calls”: the goal of replicating the dispatch jump “is to expose more information to the hardware branch predictor (specifically … the branch target predictor, which tries to guess the destination of indirect branches),” but “the C compiler doesn’t know that … and may choose to merge them together … thus undoing our careful work.” The issue author measured the damage directly by counting surviving DISPATCH sites in the compiled object (47 on main, fewer after merging) and recovered a ~1.03× speedup by inserting empty __asm__ volatile optimization barriers to keep the sites distinct. The 3.14 eval loop also carries pragmas like DONT_SLP_VECTORIZE for similar hot-loop-protection reasons. The lesson: the benefit of threaded code depends on the compiler keeping the sites distinct, which is not guaranteed and must be actively defended.

What happened to the PREDICT macros

Before the specializing interpreter, CPython had a second, complementary trick: opcode prediction via PREDICT(op) / PREDICTED(op) macros. When one opcode was very commonly followed by a specific next opcode (the classic COMPARE_OP → POP_JUMP_IF_FALSE pair), PREDICT emitted an inline check at the end of the first handler that, on a match, jumped directly to the second handler’s label without going back through the dispatch table at all — saving the decode and the indirect jump entirely. This was effectively hand-coded super-instruction chaining.

Resolved (2026-06-01)

The PREDICT/PREDICTED opcode-pair-prediction machinery is fully gone in 3.14, removed in two stages. The PREDICT(op) emitter and all its call sites disappeared in 3.13: grep PREDICT Python/bytecodes.c returns 0 at the v3.13.0 tag (vs 11 at v3.12.0, where Python/ceval_macros.h still #defined the macros). A couple of vestigial label macros (PREDICTED, PREDICT_ID, used by GO_TO_INSTRUCTION) lingered in v3.13.0’s ceval_macros.h, but by v3.14.5 even those are absent (grep "define PREDICT" Python/ceval_macros.h → 0; grep PREDICT Python/bytecodes.c → 0). The JUMP_TO_PREDICTED macro that does appear in v3.14.5 ceval_macros.h is unrelated — it belongs to the tail-call interpreter’s branch-side dispatch, not the old opcode-pair prediction. The mechanism was subsumed by the specializing adaptive interpreter, exactly as expected.

The Tail-Call Interpreter — a Third Dispatch Strategy (3.14)

Python 3.14 introduces a fundamentally different dispatcher that reaches the same per-site-prediction goal without goto. Instead of one big function with labels, each opcode is its own C function, and the dispatcher chains them with guaranteed tail calls: a handler finishes by returning a call to the next opcode’s handler, and a compiler annotation forces that call to be a true tail call (reusing the stack frame) rather than a recursion that would blow the stack (What’s New in 3.14).

The What's New document describes it as an interpreter that “uses tail calls between small C functions implementing individual Python opcodes, rather than one large C case statement.” The macros from ceval_macros.h show the mechanism: under Py_TAIL_CALL_INTERP, DISPATCH_GOTO() becomes Py_MUSTTAIL return (INSTRUCTION_TABLE[opcode])(TAIL_CALL_ARGS), where Py_MUSTTAIL expands to [[clang::musttail]] — Clang’s attribute that guarantees the call is compiled as a tail call or fails to compile. The handlers are declared Py_PRESERVE_NONE_CC, a calling convention (__attribute__((preserve_none))) that frees the compiler from preserving callee-saved registers across these calls, making the per-opcode dispatch cheaper still. The frame state (frame, stack_pointer, next_instr, oparg) is threaded through as function arguments (TAIL_CALL_PARAMS) rather than living in locals.

Why does this give the same benefit as computed gotos? Each opcode being its own function means each return next_handler(...) is, after musttail, a distinct indirect jump with its own BTB site — the same per-opcode-history property, achieved through the function-call boundary instead of a replicated goto. It can also produce better register allocation, because the compiler optimizes each small handler independently rather than juggling one enormous function.

Requirements and the benchmark caution

The tail-call interpreter requires Clang 19 or newer on x86-64 or AArch64 (GCC support is expected later), and is opt-in at build time via --with-tail-call-interp; profile-guided optimization is strongly recommended alongside it (What’s New in 3.14). It is an internal implementation detail and changes no observable Python behavior.

Its performance history is a cautionary tale worth knowing. The initial 2025 announcement reported a startling ~9–15% speedup, but that turned out to be largely an artifact: the computed-goto baseline it was compared against had been slowed by a regression bug in Clang 19. Ken Jin, the author, publicly corrected the record: “The real performance uplift one can expect by upgrading to the tail-calling interpreter is between the 3–5% range” once the baseline was fixed (Ken Jin, “I’m Sorry for Python’s tail-calling Interpreter’s Results”; independent analysis in Nelson Elhage, “Performance of the Python 3.14 tail-call interpreter”). The What's New text reflects the corrected figure: “a geometric mean of 3–5% faster on the standard pyperformance benchmark suite … The baseline is Python 3.14 built with Clang 19.” The takeaway is double: the tail-call interpreter is a modest improvement over already-good computed gotos, and — more broadly — measuring interpreter dispatch performance is treacherous because the compiler can quietly help or hurt either side. See The Tail-Call Interpreter for the full treatment.

Failure Modes and Gotchas

Compiler merges the dispatch sites. As above, tail-merging can collapse computed gotos into a switch-equivalent, silently erasing the benefit. Symptom: no speedup from USE_COMPUTED_GOTOS=1. CPython mitigates with anti-merging pragmas; third parties building CPython with aggressive optimizers can hit this (gh-129987).

Assuming computed gotos are always on. They require compiler support; MSVC historically lacked labels-as-values, so Windows/MSVC builds used the switch dispatcher. Don’t assume a given CPython binary is “threaded.”

Confusing dispatch with specialization. Computed gotos make every dispatch cheaper; specialization (PEP 659) makes the handlers cheaper by swapping in type-specific opcodes. They are orthogonal layers — a specialized opcode is still dispatched through the same opcode_targets[] table. Specialization is The Specializing Adaptive Interpreter, not this note.

“Tail call = tail-call optimization of Python functions.” No. The tail-call interpreter is about how C-level opcode handlers chain; it does not give Python-level tail-call optimization (recursive Python functions still grow the call stack). The docs call this out explicitly (What’s New in 3.14).

Alternatives and When Each Applies

The three strategies are not user-selectable per program — they are build-time properties of the CPython binary. Switch is the universal fallback, used wherever the compiler lacks labels-as-values (notably MSVC). Computed gotos are the default on GCC/Clang and have been the standard production dispatcher since Python 3.2. The tail-call interpreter is the experimental opt-in for Clang-19+ builds chasing the last few percent, best paired with PGO. For nearly everyone, “computed gotos, on by default” is the operative reality.

See Also

• The CPython Evaluation Loop — the for(;;) loop that hosts the dispatch
• Python Bytecode Instruction Set — the instructions being dispatched, and NEXTOPARG decoding
• The Tail-Call Interpreter — the 3.14 dispatch strategy in depth, with the full benchmark story
• The Specializing Adaptive Interpreter — orthogonal optimization of the handlers, not the dispatch
• The Value Stack and Frame Evaluation — what the handlers operate on between dispatches
• The eval Breaker — the periodic check that interrupts the dispatch loop
• Python Internals MOC — §3 The Bytecode Interpreter