sys.monitoring

sys.monitoring is CPython’s low-overhead execution-event monitoring API, added in Python 3.12 by PEP 669 (sys.monitoring docs, PEP 669). It exists to solve the problem that doomed sys.settrace: a debugger or coverage tool no longer has to pay a Python callback on every line of every frame. Instead a tool subscribes to specific events (function start, return, line, call, branch, raise, …), CPython instruments only the code objects that need it by rewriting their bytecode in place, and — the key trick — a callback can return the DISABLE sentinel to permanently switch off instrumentation at that one location, so cold code costs nothing after its first event. PEP 669’s design goal was that “code run under a debugger on 3.12 should outperform code run without a debugger on 3.11.” It is the engine that sys.settrace itself now runs on, and the default measurement core of modern coverage.py.

Mental Model

The old settrace model is a single phone line you can never hang up: once a frame is traced, you are called on every line forever. sys.monitoring is the opposite — a pay-per-event subscription with an unsubscribe button at each location. You register as one of up to six “tools.” You declare exactly which event kinds you care about (a bitmask). CPython then edits the bytecode of the relevant code objects, replacing ordinary opcodes with INSTRUMENTED_* variants that, when executed, fire your callback. And whenever your callback decides it has learned all it needs from a given (code object, bytecode offset) location, it returns DISABLE and CPython de-instruments that one spot — restoring the normal opcode — until you explicitly call restart_events(). The cost therefore scales with the number of distinct locations you still care about, not with how many times they execute.

flowchart TD
    A["use_tool_id(id, name)<br/>claim one of 6 tool slots (0-5)"] --> B["register_callback(id, EVENT, fn)"]
    B --> C["set_events(id, mask)  // global<br/>or set_local_events(id, code, mask)"]
    C --> D["CPython rewrites bytecode:<br/>opcode → INSTRUMENTED_opcode<br/>per code object"]
    D --> E{"Instrumented opcode<br/>executes"}
    E --> F["callback fn(code, offset, ...)"]
    F -->|returns DISABLE| G["de-instrument THIS<br/>(code, offset) location<br/>→ restore normal opcode"]
    F -->|returns anything else| H["keep instrumentation;<br/>fire again next time"]
    G -.->|"restart_events()"| D

Figure: the lifecycle of a monitored location. The decisive edge is the DISABLE branch — it is what turns “instrument once, observe once, pay nothing thereafter” into the default behaviour, and it is exactly the lever settrace cannot pull because its contract requires seeing every line every time.

Why It Exists

PEP 669 opens with the blunt diagnosis: “Using a profiler or debugger in CPython can have a severe impact on performance. Slowdowns by an order of magnitude are common” (PEP 669). The aspiration was equally blunt: “C++ and Java developers expect to be able to run a program at full speed (or very close to it) under a debugger. Python developers should expect that too.” The legacy settrace mechanism cannot deliver this for a structural reason — its semantics demand a Python call on every executed line, and it provides no per-location off-switch. PEP 669 builds on the quickening machinery introduced by PEP 659 (the specializing adaptive interpreter), which already rewrites bytecode at runtime; monitoring piggybacks on the same in-place-rewrite capability to insert instrumentation surgically rather than flipping the whole interpreter onto a slow path.

Tool Identifiers

The virtual machine supports a fixed number of monitoring tools running simultaneously, each identified by a small integer so they cannot clobber one another’s instrumentation. Python code may use tool IDs 0–5 inclusive — six tools. Four IDs are predefined by convention so unrelated tools don’t fight over the same slot (sys.monitoring docs):

sys.monitoring.DEBUGGER_ID  = 0
sys.monitoring.COVERAGE_ID  = 1
sys.monitoring.PROFILER_ID  = 2
sys.monitoring.OPTIMIZER_ID = 5

A tool claims and releases its ID through:

• use_tool_id(tool_id, name) — claim an ID; raises ValueError if it is already in use. Verified on 3.14.5: use_tool_id(6, "x") raises ValueError: invalid tool 6 (must be between 0 and 5).
• get_tool(tool_id) — returns the registered name, or None if the slot is free.
• clear_tool_id(tool_id) — unregisters all of that tool’s events and callbacks.
• free_tool_id(tool_id) — releases the ID (calling clear_tool_id first).

There is a subtlety worth pinning down precisely because it is the bridge to the sibling note: the internal VM actually reserves eight tool slots. Include/internal/pycore_instruments.h (v3.14.5) defines PY_MONITORING_TOOL_IDS 8, with IDs 6 (PY_MONITORING_SYS_PROFILE_ID) and 7 (PY_MONITORING_SYS_TRACE_ID) reserved for the C implementation of [[sys.settrace and Tracing Hooks|sys.setprofile and sys.settrace]]. Those two are not reachable from Python — use_tool_id enforces the 0–5 range — which is exactly how the legacy hooks coexist with user monitoring tools without contention.

The Event Set

Events live in the sys.monitoring.events namespace as power-of-two integer constants, combined with bitwise OR. Verified live on 3.14.5 — e.g. PY_START == 1, PY_RETURN == 4, CALL == 16, LINE == 32, INSTRUCTION == 64, BRANCH_LEFT == 256, BRANCH_RIGHT == 512, RAISE == 2048, and NO_EVENTS == 0. The events fall into three groups by how they can be controlled (sys.monitoring docs, PEP 669):

Local events — tied to a specific bytecode location, individually disable-able via DISABLE:

• PY_START — a Python function begins (callee’s frame already on the stack).
• PY_RESUME — a generator/coroutine resumes (except via throw()).
• PY_RETURN — a Python function is about to return; callback gets the return value.
• PY_YIELD — a Python function is about to yield; callback gets the yielded value.
• CALL — a call is about to happen in Python code.
• LINE — execution reaches an instruction whose source line differs from the previous instruction’s.
• INSTRUCTION — a single VM instruction is about to execute (the finest grain; high cost).
• JUMP — an unconditional control-flow jump.
• BRANCH_LEFT / BRANCH_RIGHT — a conditional branch took its left / right edge.
• STOP_ITERATION — an artificial StopIteration at the end of an iterator.

Ancillary events — controlled by CALL, observed only if CALL is monitored:

• C_RETURN — return from a non-Python callable (built-in/extension).
• C_RAISE — exception raised from a non-Python callable.

Other events — not tied to a single location, cannot be individually disabled with DISABLE:

• RAISE — an exception is raised (except those that become STOP_ITERATION).
• RERAISE — an exception is re-raised (e.g. at the end of a finally).
• EXCEPTION_HANDLED — an exception is caught.
• PY_UNWIND — a Python function exits during exception unwinding.
• PY_THROW — a Python function is resumed via throw().

The BRANCH split — a 3.13/3.14 evolution

The original PEP 669 had a single BRANCH event. In CPython 3.14 it was deprecated in favour of two events, BRANCH_LEFT and BRANCH_RIGHT — the whatsnew text (v3.14.5 source) reads: “Add two new monitoring events, BRANCH_LEFT and BRANCH_RIGHT. These replace and deprecate the BRANCH event.” The 3.14 docs add the why: “Using BRANCH_LEFT and BRANCH_RIGHT events will give much better performance as they can be disabled independently.” The point is concrete: with a single BRANCH event, a branch that has been taken in both directions can never be disabled (you still need to know about the other direction), so it keeps firing forever. Splitting into two events keyed at the same offset but distinguished by direction lets a coverage tool DISABLE each direction independently the moment it has seen it — once a branch has gone both ways, both events are off and the location is free (discuss.python.org thread). This is what made fast branch coverage finally practical. (BRANCH itself still exists in the namespace on 3.14.5 for compatibility, alongside the new pair — confirmed live.) The other monitoring addition between releases was on the C side: Python 3.13 added a suite of C-API functions for generating PEP 669 events from extension code (the PyMonitoring_* family, documented under the C-API monitoring section), so C extensions and embedders can emit the same events as the Python-level API.

There is one more performance-driven quirk, STOP_ITERATION: PEP 380 says returning from a generator raises StopIteration, but CPython 3.12+ avoids materializing that exception when it would not be observable. The docs note the STOP_ITERATION event is provided so tools can monitor real exceptions without forcing generators to pay for an exception object, and that STOP_ITERATION and a RAISE of StopIteration are treated as interchangeable, with CPython favouring STOP_ITERATION for speed.

Callback Registration and the Event Bitmask

Two orthogonal things must both be true for a callback to fire: the event must be turned on, and a callback must be registered (sys.monitoring docs). They are separate calls:

• register_callback(tool_id, event, func) — bind func to one event for one tool. Returns the previously registered callback (or None); pass func=None to unregister.
• set_events(tool_id, event_set) — turn on events globally (across all code). “No events are active by default.”
• set_local_events(tool_id, code, event_set) — turn on events for one code object only — the surgical, cheap form.
• get_events / get_local_events — read back the current masks.

An event fires once even if enabled both globally and locally. Each event’s callback has a fixed signature; the first argument is always the code object, and most include the bytecode instruction_offset:

event(s)	callback signature
PY_START, PY_RESUME, INSTRUCTION	func(code, instruction_offset)
PY_RETURN, PY_YIELD	func(code, instruction_offset, retval)
CALL, C_RAISE, C_RETURN	func(code, instruction_offset, callable, arg0)
RAISE, RERAISE, EXCEPTION_HANDLED, PY_UNWIND, PY_THROW, STOP_ITERATION	func(code, instruction_offset, exception)
LINE	func(code, line_number)
JUMP, BRANCH_LEFT, BRANCH_RIGHT	func(code, instruction_offset, destination_offset)

For CALL, if the callable takes no positional argument, arg0 is the sentinel sys.monitoring.MISSING; for an instance method, callable is the function found on the class and arg0 is self (sys.monitoring docs). Note LINE is the one event whose callback receives a line number, not an offset — a small asymmetry that exists because line tracking is the historically central use.

The DISABLE Sentinel — the source of the speed

sys.monitoring.DISABLE is “the key to near-zero overhead on cold code.” When a callback returns it, CPython removes that specific (code, instruction_offset) location’s instrumentation — the docs: “If a callback function returns DISABLE, then that function will no longer be called for that (code, instruction_offset) until sys.monitoring.restart_events() is called.” The docs are emphatic about why this matters: “Disabling events for specific locations is very important for high performance monitoring. For example, a program can be run under a debugger with no overhead if the debugger disables all monitoring except for a few breakpoints.” DISABLE only works for local events; returning it from a global-event-only callback raises ValueError. The complementary restart_events() re-enables everything that was disabled, for all tools — used when a tool needs a fresh measurement pass.

A worked example, verified live on 3.14.5:

import sys
mon = sys.monitoring
E = mon.events
 
TID = 3                                  # claim a user tool slot (0-5)
mon.use_tool_id(TID, "demo")
 
hits = []
def on_line(code, line_no):
    hits.append((code.co_name, line_no))
    return mon.DISABLE                   # observe this line once, then stop paying for it
 
def target(n):
    total = 0
    for i in range(n):
        total += i
    return total
 
mon.register_callback(TID, E.LINE, on_line)   # bind callback to LINE
mon.set_local_events(TID, target.__code__, E.LINE)  # instrument ONLY target's code object
target(3)
target(3)                                # second call: lines already DISABLEd → no callbacks
print(hits)
mon.free_tool_id(TID)

Line-by-line: use_tool_id(TID, "demo") claims slot 3. register_callback binds on_line to the LINE event for that tool. set_local_events(TID, target.__code__, E.LINE) instruments only target’s code object — target’s LINE opcodes are rewritten to INSTRUMENTED_LINE; nothing else in the program is touched. The first target(3) fires on_line once per distinct line offset, each of which returns DISABLE, de-instrumenting that location. The second target(3) produces no callbacks at all — the contrast with settrace, which would re-fire every line on every call. (One subtlety the run exposes: the for line reports twice because the loop has two distinct instrumented offsets — the loop setup and the FOR_ITER landing point — and DISABLE is keyed on (code, offset), so each is disabled independently. The same “disable at the finest meaningful granularity” principle — applied per branch direction rather than per offset — is what motivated the BRANCH_LEFT/BRANCH_RIGHT split discussed below.)

How It Works — Per-Code-Object Bytecode Rewriting

The mechanism is bytecode rewriting per code object, and the data structure lives on the code object itself. Include/cpython/code.h (v3.14.5) shows the PyCodeObject carries struct _PyCoMonitoringData *_co_monitoring; /* Monitoring data */ and uintptr_t _co_instrumentation_version; /* current instrumentation version */, plus a CO_NO_MONITORING_EVENTS flag for opting out. When a tool enables an event for a code object, CPython walks that object’s bytecode and replaces each relevant opcode with its INSTRUMENTED_* counterpart — CALL → INSTRUMENTED_CALL, RETURN_VALUE → INSTRUMENTED_RETURN_VALUE, and so on; the original opcode is stashed in _co_monitoring so it can be restored. The _co_instrumentation_version is a monotonically advancing stamp: when the global monitoring state changes, a code object’s stale local instrumentation is detected by a version mismatch and rebuilt lazily. Events that have no single fixed location — RAISE, EXCEPTION_HANDLED — are handled by runtime checks in the exception machinery rather than by a rewritten opcode.

This is where monitoring touches the specializing adaptive interpreter and instruction dispatch. An INSTRUMENTED_* opcode and a specialized opcode (e.g. BINARY_OP_ADD_INT) are mutually exclusive — a bytecode slot can be one or the other. So enabling monitoring on a code object forces its specialized instructions back to generic forms and invalidates any JIT executors built over that code; CPython’s instrumentation routines call _Py_Executors_InvalidateDependency() / _Py_Executors_InvalidateAll() to do so. The dispatch loop (The CPython Evaluation Loop) then routes the instrumented opcodes through the monitoring callback path. The crucial difference from settrace is scope: because monitoring instruments per code object and DISABLE removes instrumentation at the offset granularity, the de-specialization penalty is confined to the locations a tool actually cares about, instead of the whole program.

settrace Now Runs On This

The most striking demonstration of the design’s generality is that sys.settrace and sys.setprofile are themselves implemented on top of sys.monitoring as of 3.12. Python/legacy_tracing.c (v3.14.5) is literally titled “Support for legacy tracing on top of PEP 669 instrumentation” — it registers internal C callbacks on the reserved tool IDs 6 (profile) and 7 (trace), subscribes them to monitoring events globally, and forwards each PEP 669 event to the old (frame, event, arg) callback. Confirmed by source inspection: c_tracefunc no longer appears in Python/ceval.c or the generated opcode handlers — the legacy hook is now just another monitoring consumer that, by its semantics, can’t use DISABLE broadly. See sys.settrace and Tracing Hooks for the full story of why that makes it slow.

Failure Modes and Common Misunderstandings

• Registering a callback is not enough. You must both register_callback and set_events/set_local_events. Forgetting the latter is the most common “my callback never fires” bug.
• Returning DISABLE from a global-only event raises. DISABLE is a local-event mechanism; on a globally-enabled event with no specific location it raises ValueError. Use it for LINE/CALL/BRANCH_*, not for RAISE/EXCEPTION_HANDLED.
• DISABLE is sticky until restart_events(). A tool that disables a location and then expects to see it again on the next pass must call restart_events(); otherwise it stays silent.
• Tool-ID collisions. Two libraries both grabbing DEBUGGER_ID = 0 will conflict — use_tool_id raises if the slot is taken. Cooperative tools should claim the conventional ID for their role or pick a free one.
• Using the deprecated BRANCH. On 3.14 prefer BRANCH_LEFT/BRANCH_RIGHT; BRANCH still works but is deprecated and slower because its two directions can’t be disabled independently.
• Instrumentation cost is not zero, just lazy. Heavy events like INSTRUCTION or never-DISABLEd LINE callbacks still de-specialize and still cost per execution — sys.monitoring is cheap when you use DISABLE, not unconditionally.

Reviewed 2026-06-01

Spot-checked against Python/instrumentation.c at the v3.14.5 tag: the EVENT_FOR_OPCODE table (lines 77–84) and the INSTRUMENTED_OPCODES/_PyOpcode_Deopt mapping (around lines 116–146, 773–781) confirm the event→opcode wiring and the de-specialization-on-instrument behaviour described above — instrumenting takes a specialized opcode back to its base form, swaps in the INSTRUMENTED_* variant, and resets the adaptive counter. The full per-version reconciliation algorithm (_co_instrumentation_version mismatch handling) was not transcribed line by line, but nothing in the read contradicts the prose; this is left as an exhaustive-read item rather than an open accuracy doubt.

Alternatives and When to Choose Them

Need	Use	Why
Always-on, low-overhead instrumentation; only specific events	sys.monitoring	DISABLE → pay only for live locations; per-code-object scope
Observe every line, broad implementation compatibility, simplicity	sys.settrace and Tracing Hooks	Universal, but ~order-of-magnitude slowdown
Call/return profiling on the cheap	sys.setprofile / Profiling with cProfile	No per-line cost
Zero in-process overhead / can’t modify target	Sampling Profilers and py-spy	Samples another process’s stacks
Attach to a running process	Remote Debugging in CPython (PEP 768)	No pre-installed hook required

Choose sys.monitoring for any tool that must run continuously or over large workloads — production profilers, coverage at scale, always-on tracers. It is, as of 3.14, the recommended foundation for new tools; settrace remains only for maximum compatibility or quick one-offs.

Production Notes

The flagship adopter is coverage.py. Its sysmon core — built directly on sys.monitoring — became the default on Python 3.14+, where the BRANCH_LEFT/BRANCH_RIGHT split finally let it measure branch coverage cheaply (on 3.12/3.13 the sysmon core supported only line coverage and fell back to the older ctrace core for branches) (Batchelder 2025). Batchelder’s write-up is a candid record of the API maturing release-over-release: the single BRANCH event was workable but couldn’t be disabled once a branch went both ways, and the 3.14 split was driven substantially by coverage.py’s needs (discuss.python.org). The lesson for adopters: sys.monitoring is powerful but still young — feature coverage and event semantics genuinely shifted between 3.12, 3.13, and 3.14, so version-gate carefully and test against each.

See Also

• sys.settrace and Tracing Hooks — the legacy hook this API replaced and now powers
• The Specializing Adaptive Interpreter — the quickening machinery monitoring rewrites on top of, and what it de-optimizes
• Instruction Dispatch and Computed Gotos — how INSTRUMENTED_* opcodes are dispatched
• The CPython Evaluation Loop · Code Objects — the loop and the object that carries _co_monitoring
• The pdb Debugger · Profiling with cProfile · Sampling Profilers and py-spy · Remote Debugging in CPython
• Python Internals MOC §17 — Observability, Debugging, and Profiling