vault

Cell Variables and Closures

16 min read

Cell Variables and Closures

A closure is a function that survives the disappearance of the scope it was defined in and yet still reads and writes the local variables of that vanished scope. CPython implements this with a one-slot heap box called a cell (PyCellObject in C). When the compiler sees that an inner function references a variable belonging to an enclosing function, it promotes that variable from an ordinary fast local into a cell: instead of storing the value in the enclosing frame’s local-variable slot, it stores a pointer to a cell, and the inner function is handed a reference to the same cell. Both frames now read and write through one shared box, so a mutation made by the inner function is visible to the outer one and vice-versa, and the box outlives the outer frame for exactly as long as some closure still references it. The data model puts it precisely: a function’s __closure__ is “None or a tuple of cells that contain bindings for the names specified in the co_freevars attribute of the function’s code object” (per the data model reference). This note is the mechanism; the famous loop-variable surprise that this machinery produces lives in Late Binding Closures.

The Problem: A Local That Must Outlive Its Frame

Ordinary Python locals live in their frame and die with it. When a function returns, its frame is torn down and every local-variable slot is released. That is exactly the behaviour you want for the overwhelming majority of variables — they are scratch space for one call. The model is described by the fast-locals array: each local is a fixed slot in a per-frame C array, addressed by a small integer index, and LOAD_FAST/STORE_FAST read and write that slot directly.

A closure breaks this model. Consider the canonical example:

def make_adder(x):
    def add(y):
        return x + y
    return add
 
add5 = make_adder(5)
print(add5(3))   # 8

When make_adder(5) returns, its frame is destroyed — but the function object add that it returned still needs to read x. The value 5 cannot live in make_adder’s frame, because that frame no longer exists by the time add5(3) runs. Nor can it live as a plain constant baked into add, because each call to make_adder with a different x must produce an add that closes over a different binding. And critically, x is not read-only in the general case: an inner function declared nonlocal x may reassign it, and that reassignment must be seen by every other function closing over the same x. So the requirement is precise: a single, shared, mutable binding, allocated somewhere that survives the enclosing frame and is reachable from every function that closes over it.

The terminology, fixed by the data model and the glossary: in make_adder, x is a cell variable — a local that is “referenced from at least one nested scope inside the function” (co_cellvars). In add, x is a free variable (also called a closure variable) — a name “that a nested scope references in an outer scope” (co_freevars). The same underlying box is a cell-var to the scope that owns it and a free-var to the scope that borrows it.

The Solution: A Shared One-Slot Box

CPython’s answer is the cell object. Its C definition is almost insultingly small (Include/cpython/cellobject.h):

typedef struct {
    PyObject_HEAD
    /* Content of the cell or NULL when empty */
    PyObject *ob_ref;
} PyCellObject;

That is the entire structure: the standard PyObject_HEAD (the reference count and type pointer every CPython object carries) followed by a single pointer, ob_ref, to whatever the cell currently holds — or NULL if the cell is empty. A cell is a heap-allocated box with exactly one slot. The value 5 is not stored in any frame; it is stored in a cell on the heap, and every frame that cares about x holds a reference to that one cell. Because cells are heap objects under the reference-counting system, the cell lives precisely as long as something references it — the enclosing frame while it runs, and the closure’s __closure__ tuple after the enclosing frame is gone. make_adder’s frame can die; the cell does not, because add5.__closure__[0] still points at it.

You can see all of this from Python:

>>> add5 = make_adder(5)
>>> add5.__closure__
(<cell at 0x...: int object at 0x...>,)
>>> add5.__closure__[0].cell_contents
5
>>> make_adder.__code__.co_cellvars     # x is a cell in the outer scope
('x',)
>>> add5.__code__.co_freevars           # x is free in the inner scope
('x',)

The cell exposes one Python-visible attribute, cell_contents, implemented by the getset descriptor cell_get_contents; reading it on an empty cell raises ValueError: Cell is empty (cellobject.c). The cell type even participates in the cyclic garbage collector — it defines tp_traverse/tp_clear (cell_traverse/cell_clear) and sets Py_TPFLAGS_HAVE_GC, because a cell can hold a reference that participates in a reference cycle (a function closing over a cell that, transitively, refers back to the function).

graph LR
    subgraph outer["make_adder frame (transient)"]
        OS["localsplus slot for x<br/>(holds the cell, not 5)"]
    end
    subgraph cell["PyCellObject on the heap"]
        CB["ob_ref → int 5"]
    end
    subgraph fn["add5 = PyFunctionObject"]
        CL["__closure__ = (cell,)"]
    end
    subgraph inner["add5() frame (later call)"]
        IS["free-var slot for x<br/>(holds the SAME cell)"]
    end
    OS -->|references| cell
    CL -->|references| cell
    IS -->|references| cell
    CB -.->|"LOAD_DEREF reads through it"| inner

    style cell fill:#fdf6e3,stroke:#b58900

Figure: the cell (centre) is the single shared box. The outer frame’s local slot, the closure tuple on the function object, and the inner frame’s free-variable slot all reference the same PyCellObject. The insight: there is exactly one box, not a copy per frame — which is why a nonlocal reassignment in the inner function is visible to the outer scope, and why the box (and the 5 it holds) survives the destruction of make_adder’s frame, kept alive by add5.__closure__.

How the Compiler Decides: The Symbol Table

Nothing about cells is a runtime decision — it is settled entirely at compile time by symbol-table analysis. As the compiler walks each scope it classifies every name. A name assigned in a function and also referenced by some nested function is classified CELL; a name referenced in a function but bound in some enclosing function is classified FREE. (Names bound at module level are GLOBAL; truly undefined names fall through to builtins. Crucially, “references to global and builtin names are not included” in co_freevars — only enclosing function scopes create cells, per the data model.)

This classification is what makes the difference between the two below:

def f():
    x = 1
    def g():
        return x       # g references x → x is CELL in f, FREE in g
    return g
 
def h():
    x = 1
    return x           # x referenced only locally → plain FAST local, no cell

In f, the mere existence of g’s reference to x forces x to be a cell — even before g is ever called, even if g is never called. The cost is paid at the definition site, not the use site. This is why adding a single inner reference to an outer local silently changes that local’s storage class from a fast slot to a heap-allocated cell.

The Bytecode Mechanism, Opcode by Opcode

Since CPython 3.11, local variables, cell variables, and free variables all live in one unified per-frame array called localsplus, laid out in the order [plain locals | cell variables | free variables]. The code object’s co_nlocalsplus counts the total, and co_nfreevars counts the trailing free-variable region. Every opcode that touches any of these uses an oparg that indexes this single array. (Before 3.11 cells and free variables lived in separate per-frame arrays addressed by their own opcodes; the unification into one localsplus array arrived with the 3.11 frame restructuring — see Stack Frames and the Frame Stack and Fast Locals and the LOAD_FAST Family.) Four opcodes do the work; their bodies below are from Python/bytecodes.c at v3.14.5.

MAKE_CELL — build the box at frame setup

inst(MAKE_CELL, (--)) {
    PyObject *initial = PyStackRef_AsPyObjectBorrow(GETLOCAL(oparg));   // 1
    PyObject *cell = PyCell_New(initial);                              // 2
    if (cell == NULL) { ERROR_NO_POP(); }
    _PyStackRef tmp = GETLOCAL(oparg);                                  // 3
    GETLOCAL(oparg) = PyStackRef_FromPyObjectSteal(cell);              // 4
    PyStackRef_XCLOSE(tmp);                                            // 5
}

MAKE_CELL runs once per cell variable, in the function prologue before the body executes. Line 1 reads whatever already occupies localsplus[oparg] — usually NULL, but not if the cell variable is also a parameter (an argument cell), in which case the slot already holds the argument value. Line 2 allocates a fresh PyCellObject initialised with that value (PyCell_New calls Py_XNewRef, so an argument’s value is moved into the box). Lines 3–5 swap the cell into the slot and drop the old contents. After MAKE_CELL, localsplus[oparg] holds the cell, not the value. The compiler does not emit these by hand in the body; insert_prefix_instructions in Python/flowgraph.c injects one MAKE_CELL per cell at the very front of the entry block. You can see it in disassembly as the leading line with no source-line number:

  --           MAKE_CELL                0 (x)
   4           RESUME                   0
   ...

STORE_DEREF and LOAD_DEREF — write and read through the cell

inst(STORE_DEREF, (v --)) {
    PyCellObject *cell = (PyCellObject *)PyStackRef_AsPyObjectBorrow(GETLOCAL(oparg));
    PyCell_SetTakeRef(cell, PyStackRef_AsPyObjectSteal(v));
}
 
inst(LOAD_DEREF, ( -- value)) {
    PyCellObject *cell = (PyCellObject *)PyStackRef_AsPyObjectBorrow(GETLOCAL(oparg));
    value = _PyCell_GetStackRef(cell);
    if (PyStackRef_IsNull(value)) {
        _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg);
        ERROR_IF(true);
    }
}

These are the cell counterparts of STORE_FAST/LOAD_FAST. The “DEREF” name is literal: the slot holds a cell, so the opcode first fetches the cell, then dereferences it. STORE_DEREF writes the new value into the cell’s ob_ref via PyCell_SetTakeRef (which drops the old contents); LOAD_DEREF reads ob_ref, and if it is NULL (an empty cell — the binding has not happened yet) raises NameError/UnboundLocalError through _PyEval_FormatExcUnbound. Both work identically whether the cell is one this frame created (a cell-var) or one it inherited (a free-var) — that is the whole point of the unified array. In the counter example below, the inner inc reads and writes the shared count cell entirely through LOAD_DEREF/STORE_DEREF:

def make_counter():
    count = 0
    def inc():
        nonlocal count
        count += 1
        return count
    return inc
make_counter:
  --   MAKE_CELL    1 (count)
   4   LOAD_SMALL_INT 0
       STORE_DEREF  1 (count)      # outer initialises the cell
   ...
inc:
  --   COPY_FREE_VARS 1
   7   LOAD_DEREF   0 (count)      # read shared cell
       ...
       STORE_DEREF  0 (count)      # write shared cell — visible to outer

LOAD_CLOSURE — push the cell itself to build the closure

When the outer scope builds the inner function, it must hand the inner function the actual cell objects, not their contents. That is what LOAD_CLOSURE means at the codegen level. In Python/codegen.c, codegen_make_closure emits one LOAD_CLOSURE per free variable of the inner code object, then BUILD_TUPLE, then MAKE_FUNCTION, then SET_FUNCTION_ATTRIBUTE with the closure flag:

for (; i < co->co_nlocalsplus; ++i) {
    /* Bypass com_addop_varname because it will generate
       LOAD_DEREF but LOAD_CLOSURE is needed. */
    ...
    ADDOP_I(c, loc, LOAD_CLOSURE, arg);
}
flags |= MAKE_FUNCTION_CLOSURE;
ADDOP_I(c, loc, BUILD_TUPLE, co->co_nfreevars);

The comment is the key: ordinary variable loading would emit LOAD_DEREF (push the contents), but here we need LOAD_CLOSURE (push the cell). The subtlety — and a place the brief’s framing needs correcting against observed bytecode — is that LOAD_CLOSURE is a pseudo-instruction. In Python/bytecodes.c it is declared pseudo(LOAD_CLOSURE, (-- unused)) = { LOAD_FAST, } — meaning it is resolved during assembly into a plain LOAD_FAST. This works precisely because the cell already lives in the localsplus array exactly like any fast local: pushing “the cell” is just pushing the slot’s contents, which by this point is the cell (after MAKE_CELL ran). So LOAD_CLOSURE is a compile-time intent marker, not a runtime opcode. In 3.14 disassembly it shows up resolved even further, as LOAD_FAST_BORROW (a non-reference-incrementing fast load — see Fast Locals and the LOAD_FAST Family):

make_adder:
  --   MAKE_CELL          0 (x)
   5   LOAD_FAST_BORROW   0 (x)      # was LOAD_CLOSURE in codegen
       BUILD_TUPLE        1
       LOAD_CONST         0 (<code object add ...>)
       MAKE_FUNCTION
       SET_FUNCTION_ATTRIBUTE 8 (closure)
       STORE_FAST         1 (add)

COPY_FREE_VARS — pull the inherited cells into the inner frame

The other half is in the inner function’s prologue. When add5(3) is called and its frame is built, it must populate its free-variable slots with the cells stored in add5.__closure__. COPY_FREE_VARS does exactly that:

inst(COPY_FREE_VARS, (--)) {
    PyCodeObject *co = _PyFrame_GetCode(frame);
    PyFunctionObject *func = (PyFunctionObject *)PyStackRef_AsPyObjectBorrow(frame->f_funcobj);
    PyObject *closure = func->func_closure;          // the __closure__ tuple
    assert(oparg == co->co_nfreevars);
    int offset = co->co_nlocalsplus - oparg;          // start of free-var region
    for (int i = 0; i < oparg; ++i) {
        PyObject *o = PyTuple_GET_ITEM(closure, i);
        frame->localsplus[offset + i] = PyStackRef_FromPyObjectNew(o);
    }
}

It reads the closure tuple off the function object and copies each cell into the trailing free-variable region of localsplus, which begins at co_nlocalsplus - co_nfreevars. After this runs, the inner frame’s free-var slots hold the same cell objects the outer frame created — that is the moment the two scopes become connected. Like MAKE_CELL, it is injected by insert_prefix_instructions and appears as a leading line in disassembly (COPY_FREE_VARS 1), with 1 being co_nfreevars.

Putting it end to end: the outer frame runs MAKE_CELL to box its cell-vars, uses STORE_DEREF/LOAD_DEREF to touch them, and LOAD_CLOSURE(→LOAD_FAST)+BUILD_TUPLE+MAKE_FUNCTION+SET_FUNCTION_ATTRIBUTE to attach those same cells to the inner function as its __closure__. The inner frame runs COPY_FREE_VARS to import those cells, then uses LOAD_DEREF/STORE_DEREF to read and write them. One box, two frames, shared by pointer.

The Closure Shares the Live Variable, Not a Snapshot

Because there is one cell and both scopes hold the same pointer, a closure captures a binding, not a value. This is the single most consequential property of the design and the source of both its power and its most infamous gotcha. Mutation flows both ways:

def make_pair():
    n = 0
    def up():
        nonlocal n
        n += 1
        return n
    def get():
        return n
    return up, get
 
up, get = make_pair()
up(); up()
print(get())     # 2  — get() sees up()'s mutations, because they share one cell

up and get are two distinct functions, each with its own __closure__, but both closures’ single cell is the same object — make_pair built one cell for n and handed the same one to both. There is no copy. get() returns 2 because it reads the live box that up() mutated.

The flip side is the loop-variable trap: if you build a list of closures inside a loop that all close over the loop variable, they share one cell and all see its final value — they did not snapshot the value at definition time. That surprise, its diagnosis, and the idiomatic fixes (default-argument capture, functools.partial, a factory function) belong to Late Binding Closures; this note establishes only why it happens — shared live box, not snapshot. The same effect is visible even within a single comprehension that builds closures: [(lambda: r) for r in range(3)] yields three lambdas that all return 2, because the comprehension’s r is a single cell mutated through the loop. The interaction between this cell machinery and comprehension scoping is detailed in Comprehension Scoping.

Common Misunderstandings

“Closures copy the variable’s value.” No — they share the binding. A snapshot semantics would make make_pair impossible. If you genuinely want a snapshot, bind the value to a default argument (def f(x=current): ...), which evaluates current at definition time and stores it on the function object’s __defaults__, not in a cell — see Default Arguments and Late Binding.

“Only the inner function uses cells.” The outer function uses them too: in make_adder, x is stored as a cell in make_adder’s own frame for the duration of that call. The cell is shared; it is not “the inner function’s.” If the outer function reassigns x after defining the inner one, the inner one sees the new value.

“nonlocal creates the cell.” The cell is created by the symbol table classifying the name as cell/free, which happens because of the reference from the inner scope. nonlocal is required only when the inner scope assigns to the name (so the compiler treats it as the same binding rather than a new local); a read-only reference needs no nonlocal and still produces a cell. Reassigning a free variable without nonlocal makes it a new local of the inner scope instead — a classic trap covered in The Global and Nonlocal Statement Traps.

“global and nonlocal are the same mechanism.” They are not. global names are looked up by dictionary in the module’s globals (LOAD_GLOBAL/STORE_GLOBAL), never through cells. Only enclosing function scopes produce cells. See Local Global and Nonlocal Scopes.

Production Notes

Cells appear constantly in real code: every decorator that wraps a function and refers to it, every method that uses super() (which closes over an implicit __class__ cell — visible as ste_needs_class_closure in the class-body codegen), every callback or partial built with a free variable, and the body of any class that references __class__. The performance cost is real but small: cell access is an extra pointer indirection versus a plain fast local, and creating cells adds one heap allocation per cell-var per call. This is why CPython does not box every local — only those genuinely captured — and why a hot inner loop that does not close over outer locals pays nothing. The cell_contents attribute is occasionally used for introspection and debugging (func.__closure__[i].cell_contents), and writing to it is possible but exotic; libraries that “rebind” closures (some mocking and hot-reload tools) do exactly this. Under the free-threaded build, cell access is wrapped in a critical section (Py_BEGIN_CRITICAL_SECTION) and uses atomic loads, because a cell shared across closures can be touched by multiple threads concurrently — see pycore_cell.h’s PyCell_GetRef/PyCell_SwapTakeRef.

See Also

Graph View

Backlinks