vault

The Context Manager Protocol

16 min read

The Context Manager Protocol

A context manager is any object that defines a runtime context to enter before a block of code runs and to exit afterward, encapsulated as a pair of special (“dunder”) methods: __enter__(self), run on the way in, whose return value is bound to the as target; and __exit__(self, exc_type, exc_value, traceback), run on the way out — unconditionally, whether the block finished normally, raised, returned, or broke. The single defining feature of the protocol is the exit contract: __exit__ receives the live exception that is propagating out of the block (or three Nones if none is), and if it returns a true value the exception is swallowed; any false/None return re-raises it. The Python reference states it plainly: “If an exception is supplied, and the method wishes to suppress the exception (i.e., prevent it from being propagated), it should return a true value. Otherwise, the exception will be processed normally upon exit from this method” (per the data model reference). This note owns the contract — what the methods are, what they must return, and the contextlib ecosystem built on top of them. Its sibling The with Statement and Cleanup owns the bytecode and control flow — how with actually compiles and how the interpreter drives these calls.

The protocol was introduced by PEP 343 (“The ‘with’ Statement”, targeting Python 2.5, 2005), whose stated goal was “to make it possible to factor out standard uses of try/finally statements” (PEP 343). The asynchronous variant — __aenter__/__aexit__, driven by async with — arrived with PEP 492 (“Coroutines with async and await syntax”, Python 3.5). Everything below is verified against the CPython v3.14.5 documentation and standard-library source (as of 31 May 2026).

Mental Model

Think of a context manager as a balanced bracket around a block of code: __enter__ is the opening bracket (acquire the lock, open the file, begin the transaction, save the global state), and __exit__ is the matching closing bracket (release, close, commit-or-rollback, restore). The language guarantees the closing bracket fires exactly when the opening one succeeded — the same guarantee a try/finally gives you, but named once and reused everywhere. What a raw try/finally cannot do, and what makes the protocol more than syntactic sugar, is the exit method’s say over the exception: because __exit__ is handed the in-flight exception and its boolean return decides the exception’s fate, a context manager can examine an error and decide to suppress it — something a finally block can only do clumsily.

graph TD
    START["with cm as x:"] --> ENTER["x = type(cm).__enter__(cm)<br/>(return value bound to 'as' target)"]
    ENTER --> BODY["run the block body"]
    BODY -->|"normal completion / return / break"| EXITNORM["type(cm).__exit__(cm, None, None, None)<br/>return value IGNORED"]
    BODY -->|"raises EXC"| EXITEXC["type(cm).__exit__(cm, type(EXC), EXC, EXC.__traceback__)"]
    EXITEXC -->|"returns truthy"| SUPPRESS["exception SWALLOWED<br/>execution continues after with"]
    EXITEXC -->|"returns falsy / None"| RERAISE["exception RE-RAISED<br/>propagates out of with"]
    EXITNORM --> AFTER["execution continues after with"]
    SUPPRESS --> AFTER

Figure: The two exit paths of the protocol. On normal completion (left fork) __exit__ is called with three Nones and its return value is discarded — you cannot “suppress” a non-exception. On an exception (right fork) __exit__ is handed the exception triple and its truthiness is the verdict: truthy swallows, falsy re-raises. The insight to carry away: a return True written by accident in __exit__ silently eats every exception your block ever raises — the single most common context-manager bug.

The Two Methods, Precisely

__enter__(self)

__enter__ runs the setup. The reference says: “Enter the runtime context related to this object. The with statement will bind this method’s return value to the target(s) specified in the as clause of the statement, if any” (per data model). Two subtleties trip people up:

  • The as target is whatever __enter__ returns — not the context manager itself. It is conventional for __enter__ to return self (a file object opened by open() returns the file), but it is not required. open() returns the file object; contextlib.suppress() returns the suppressor (itself); a database-connection context manager might return a fresh cursor rather than the connection. Code that assumes with foo() as x: makes x is foo() is relying on convention, not the protocol. The classic gotcha is threading.Lock: with lock as x: binds x to True (the boolean the lock’s __enter__/acquire returns), not to the lock — so x is almost never useful, and you simply write with lock:.

  • If __enter__ raises, __exit__ is never called. The bracket is only “balanced” once the opening bracket succeeded. The compound-statement reference is explicit: “The with statement guarantees that if the __enter__() method returns without an error, then __exit__() will always be called” (per compound statements). The corollary — also stated there — is that an error during the assignment to the as target is treated as an error inside the body, so __exit__ does run for it, because __enter__ had already returned successfully.

__exit__(self, exc_type, exc_value, traceback)

__exit__ runs the teardown and arbitrates the exception. Verbatim from the reference: “Exit the runtime context related to this object. The parameters describe the exception that caused the context to be exited. If the context was exited without an exception, all three arguments will be None. … Note that __exit__() methods should not reraise the passed-in exception; this is the caller’s responsibility” (per data model). Three rules govern its behaviour:

  1. The three arguments are the exception triple: the exception’s class (exc_type), the instance (exc_value), and its traceback object (traceback). On a clean exit all three are None. This triple is the legacy sys.exc_info() shape, retained for the protocol even though modern CPython tracks a single exception object internally.

  2. A true return suppresses; a false/None return propagates. Crucially, a method that ends without an explicit return returns None, which is falsy — so a __exit__ that just does cleanup correctly lets exceptions through. You must deliberately return True to suppress, and you should only ever do so for an exception you actually inspected and chose to handle (see Failure Modes).

  3. Suppress by returning, never by re-raising. If __exit__ wants to replace the exception with a different one, it simply raises the new one (which propagates as normal, with the original chained via __context__); it must not re-raise the passed-in exception, because the interpreter is already prepared to re-raise it for you and doing so manually corrupts the traceback.

__enter__/__exit__ are looked up by name on the type, not via a C-level type slot — there is no tp_enter/tp_exit in CPython, unlike tp_iter for the iterator protocol. This is confirmed two ways at the v3.14.5 tag: grepping the PyTypeObject/PyHeapTypeObject struct definitions in both Include/object.h and Include/cpython/object.h for tp_enter, tp_exit, __enter__, or __exit__ yields no matches (the struct simply has no such fields); and the compiler emits LOAD_SPECIAL — a by-name special-method lookup — rather than a slot call when compiling with. The context-manager protocol is therefore implemented entirely as a by-name lookup of __enter__/__exit__ on the type, with no dedicated slot anywhere in the C type object.

Why type-level lookup matters

Both methods are implicitly invoked special methods, so they are subject to Python’s special method lookup rule: the interpreter looks them up on the object’s type, bypassing the instance __dict__ and any __getattribute__ override. The reference’s own example for this rule is, fittingly, the context-manager methods: “This ensures that special methods like __enter__() and __exit__() are guaranteed to be discovered when implementing context managers, without being affected by instance dictionaries or __getattribute__() customization” (per special method lookup). The practical upshot: you cannot make an object a context manager by stuffing __enter__/__exit__ into its instance __dict__ (e.g. instance.__exit__ = lambda *a: None); the methods must live on the class. See Dunder Methods and the Data Model for the general rule.

A Concrete Class-Based Context Manager

import time
 
class Timer:
    def __init__(self, label):
        self.label = label
 
    def __enter__(self):
        self.start = time.perf_counter()
        return self                 # bound to the `as` target
 
    def __exit__(self, exc_type, exc_value, traceback):
        elapsed = time.perf_counter() - self.start
        print(f"{self.label}: {elapsed:.4f}s")
        # implicit `return None` → falsy → exceptions propagate unchanged

Line-by-line:

  • __enter__ records a start time on self and return self, so with Timer("load") as t: binds t to the timer; a caller can read t.start inside the block.
  • __exit__ accepts the full exception triple even though this manager ignores it. It computes the elapsed time and prints it. Because it falls off the end with no return, it returns None — meaning if the block raised, the timer still prints (cleanup always runs) and the exception still propagates. That is exactly the behaviour you want for an observability wrapper: measure, then let the error fly.

A manager that does arbitrate the exception — a transaction wrapper — looks like this:

class Transaction:
    def __init__(self, conn):
        self.conn = conn
 
    def __enter__(self):
        self.conn.execute("BEGIN")
        return self.conn            # hand the caller the connection, not self
 
    def __exit__(self, exc_type, exc_value, traceback):
        if exc_type is None:
            self.conn.execute("COMMIT")
        else:
            self.conn.execute("ROLLBACK")
        return False                # never swallow — let the caller see the error

Here __exit__ branches on whether an exception occurred (exc_type is None) to commit or roll back, and explicitly return False so the error still surfaces to the caller after the rollback. This is the canonical database pattern, and it is why the protocol passes the exception to __exit__ at all: the teardown’s behaviour depends on how the block ended.

The Generator-Based Context Manager: contextlib.contextmanager

Writing a whole class for a setup/teardown pair is heavy. contextlib.contextmanager lets you write a context manager as a generator with a single yield — the code before the yield is the __enter__ body, the yielded value is the as target, and the code after the yield is the __exit__ body. This builds directly on the generator machinery covered in Generators and the yield Mechanism.

from contextlib import contextmanager
 
@contextmanager
def managed_resource(*args, **kwds):
    resource = acquire_resource(*args, **kwds)   # __enter__ body
    try:
        yield resource                           # value bound to `as`; block runs here
    finally:
        release_resource(resource)               # __exit__ body — always runs

The mechanism (per the contextlib docs): “The function being decorated must return a generator-iterator when called. This iterator must yield exactly one value, which will be bound to the targets in the with statement’s as clause.” When the with block is entered, the decorator’s wrapper object calls next() on the generator, running it up to the yield and capturing the yielded value as the __enter__ return. When the block exits normally, the wrapper calls next() again to resume the generator past the yield, running the teardown. When the block exits with an exception, the wrapper does something subtler and crucial: “If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred” — i.e. the wrapper calls generator.throw(exc), which makes the yield expression raise the block’s exception. That is precisely why you wrap the yield in try/finally (or try/except): the finally guarantees teardown, and an except lets you handle and suppress the error. If the generator swallows the thrown exception and exits cleanly, __exit__ returns true (suppressing); if the exception propagates out of the generator, it propagates out of the with. The full suppression bookkeeping lives in Lib/contextlib.py’s _GeneratorContextManager.__exit__.

This behaviour is exactly what _GeneratorContextManager.__exit__ implements in Lib/contextlib.py (verified by reading the v3.14.5 source). On a clean exit (typ is None) it calls next(self.gen), expecting a StopIteration (the generator finishing after its teardown) and returning False; if the generator yields again it raises RuntimeError("generator didn't stop"). On an exception it calls self.gen.throw(value) and then disentangles the throw-protocol from the __exit__-protocol: if the same exception comes back out (exc is value) it returns False — not suppressed, because the generator did not absorb it; only if the generator swallows the thrown exception (so throw() raises a StopIteration signalling the generator finished) does it suppress. There is careful special handling for StopIteration and RuntimeError (PEP 479) so that a StopIteration raised inside the with body is not accidentally swallowed.

The async sibling contextlib.asynccontextmanager applies the same trick to an async def generator (one with yield), producing an async with-compatible object — covered alongside async with lowering in The with Statement and Cleanup and the coroutine machinery in Coroutines and the async await Protocol.

The contextlib Toolbox

Beyond @contextmanager, the module ships several ready-made managers and combinators (all per the contextlib docs):

  • closing(thing) — returns a manager whose __exit__ calls thing.close(). Equivalent to a tiny @contextmanager with try: yield thing; finally: thing.close(). Use it to make any object with a close() method but no native with support usable in a with block, e.g. with closing(urlopen(url)) as page:.

  • suppress(*exceptions) — a manager whose __exit__ returns true iff the in-flight exception is an instance of one of the named classes, swallowing exactly those. with suppress(FileNotFoundError): os.remove(path) replaces the four-line try/except FileNotFoundError: pass. The docs note “This context manager is reentrant” (see reentrancy below). It is the rare, legitimate use of returning true from __exit__ — and it is correct precisely because it inspects the exception type before suppressing.

  • ExitStack — a manager that maintains a stack of cleanup callbacks, letting you enter a dynamic, data-driven number of context managers and guarantee they all unwind in reverse order. enter_context(cm) “Enters a new context manager and adds its __exit__() method to the callback stack. The return value is the result of the context manager’s own __enter__() method.” callback(fn, *args) registers a plain function (which “cannot suppress exceptions, as they are never passed the exception details”). pop_all() “Transfers the callback stack to a fresh ExitStack instance and returns it” without invoking anything — the idiom for committing a half-built set of resources so they survive past the current block. Because “registered callbacks are invoked in the reverse order of registration, this ends up behaving as if multiple nested with statements had been used.” The canonical use is opening a variable-length list of files:

    from contextlib import ExitStack
with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # every file closes in reverse order at block exit, even if a later open() raised

    The async analogue is AsyncExitStack, with enter_async_context and async-aware unwinding.

  • AbstractContextManager / AbstractAsyncContextManager — abstract base classes (see Abstract Base Classes) that provide a default __enter__ returning self and an abstract __exit__. Inheriting from them documents intent and lets issubclass structural checks work. The async ABC mirrors them with __aenter__/__aexit__.

  • ContextDecorator — a base that lets a context manager double as a function decorator (@cm), wrapping the whole decorated function in the context. The docs warn: “As the decorated function must be able to be called multiple times, the underlying context manager must support use in multiple with statements” — i.e. it must be reusable (below).

Reentrancy and Reusability

The contextlib docs draw a careful three-way distinction that catches people out:

  • Single-use managers can be entered once. A @contextmanager generator is single-use: the generator is exhausted after one trip, so a second with on the same instance fails. (Calling the decorated function again makes a fresh generator — that is fine.)

  • Reusable, not reentrant managers “support being used multiple times, but will fail … if the specific context manager instance has already been used in a containing with statement.” threading.Lock and ExitStack are the examples: you can use the same lock in two sequential with blocks, but nesting with lock: with lock: on a plain (non-recursive) lock deadlocks.

  • Reentrant managers “may also be used inside a with statement that is already using the same context manager” — threading.RLock, suppress(), redirect_stdout(), chdir(). The docs add a sharp caveat: “being reentrant is not the same thing as being thread safe. redirect_stdout(), for example, is definitely not thread safe.”

The lesson: reentrancy is a property of the specific manager, and you must check it before nesting a manager inside itself.

Failure Modes and Common Misunderstandings

  • Accidentally swallowing everything with return True. A __exit__ that ends with return True (or any truthy value) suppresses every exception the block raises, including KeyboardInterrupt and unrelated bugs. This is the protocol’s sharpest footgun. Cleanup-only managers must return None (just don’t return).

  • Re-raising the passed-in exception inside __exit__. Writing raise exc_value inside __exit__ to “let it propagate” is wrong twice over: the interpreter will also re-raise it (double exception), and you mangle the traceback. To propagate, return falsy. To replace, raise a new exception.

  • __enter__ raising leaves no cleanup. If acquisition is multi-step and the second step can fail, the first step’s resource leaks because __exit__ never runs. The fix is to make __enter__ itself transactional (clean up its own partial work on failure) or to use ExitStack so each acquired resource is registered the instant it succeeds.

  • Assuming as x gives you the manager. As covered above, x is __enter__’s return value. Reaching for x.some_manager_method() when __enter__ returned something else (or True, for locks) is a common confusion.

  • Async/sync mismatch. Using a plain with on an object that only defines __aenter__/__aexit__ raises TypeError (no __enter__); the reverse fails too. async with requires the async dunders; with requires the sync ones. There is no automatic bridging.

Alternatives and When to Choose Them

A bare try/finally is the lower-level primitive the protocol abstracts; reach for it directly only for one-off cleanup that is not worth naming. A class-based manager is best when the manager holds meaningful state, when both __enter__ and __exit__ are substantial, or when you want it to also be reusable/reentrant. A @contextmanager generator is best for the common case of “setup, yield, teardown” with no reuse requirement — it is the most readable. ExitStack is the choice when the number of managers is dynamic. closing/suppress are micro-conveniences for their exact niches. For async code, the __aenter__/__aexit__ family and asynccontextmanager are mandatory — a sync manager cannot await inside its setup/teardown.

See Also

Graph View

Backlinks