vault

CPython Memory Management Overview

12 min read

CPython Memory Management Overview

CPython’s memory management is a layered stack with two largely independent concerns at its heart. Vertically, every allocation flows through one of three allocator domains — raw, mem, and object — that sit on top of (and, for two of them, in front of) the system C allocator: the raw domain calls malloc() directly, while the mem and object domains route small allocations through pymalloc, an arena/pool/block sub-allocator that batches them away from the system heap (Python C-API: Memory Management; Objects/obmalloc.c, 3.14.0). Orthogonally, reclamation is two-tier: reference counting is the primary mechanism — an object is destroyed the instant its count hits zero (CPython InternalDocs/garbage_collector.md, 3.14) — and a cyclic garbage collector runs as a backstop solely to reclaim the reference cycles that counting structurally cannot (“some additional machinery is needed to clean these reference cycles,” ibid.). This note is the orientation map for §6 of the Python Internals MOC: it frames the whole stack and then hands off the deep mechanics to its siblings.

This is the memory-layer framing. Where this note says “an object dies at refcount zero,” the companion Reference Counting gives the object-model lens (the count as a property of every PyObject, deterministic destruction, the GIL connection), and Reference Counting Mechanics walks the Py_INCREF/Py_DECREF/_Py_Dealloc machine code. The three are deliberately distinct lenses on one mechanism — read this for the layout of the whole system, them for depth on each layer.

Mental Model — A Stack of Wrappers, and Two Reclaimers

Picture CPython’s heap management as a vertical stack of allocator wrappers, with a separate, orthogonal pair of reclamation mechanisms deciding when memory comes back. The allocation stack answers “where does a new object’s bytes come from?”; the reclamation pair answers “when are those bytes returned?”

flowchart TD
    subgraph ALLOC["Allocation: where bytes come from (top calls down)"]
        OBJ["Object domain<br/>PyObject_Malloc / Free"]
        MEM["Mem domain<br/>PyMem_Malloc / Free"]
        RAW["Raw domain<br/>PyMem_RawMalloc / Free"]
        PYM["pymalloc<br/>(arenas - pools - blocks,<br/>small objects under threshold)"]
        SYS["System allocator<br/>malloc / free (libc)"]
        OBJ -->|"default"| PYM
        MEM -->|"default"| PYM
        PYM -->|"request over threshold,<br/>or arena refill"| SYS
        RAW -->|"always direct"| SYS
    end
    subgraph RECLAIM["Reclamation: when bytes return (orthogonal)"]
        RC["Reference counting<br/>(PRIMARY: free at refcnt==0)"]
        GC["Cyclic GC<br/>(BACKSTOP: only reclaims cycles)"]
        RC -.->|"cannot reclaim cycles -><br/>hands the residue to"| GC
    end
    ALLOC -.->|"freed memory flows back<br/>down the same stack"| RECLAIM

Diagram: the allocation stack (left) is vertical and routes by domain — raw goes straight to libc malloc; mem and object default to pymalloc, which itself falls through to malloc for large requests. The reclamation pair (right) is orthogonal and tiered — refcounting frees almost everything immediately, the cyclic GC mops up only the cycles refcounting leaves behind. The insight to take away: “which allocator?” and “what reclaims it?” are independent axes. An object allocated by pymalloc is freed by refcounting (back to a pymalloc pool) in the common case, and only visits the GC if it is part of a cycle.

The Three Allocator Domains

CPython does not have one malloc; it has three domains, each a small table of function pointers (malloc, calloc, realloc, free) that the runtime can swap out wholesale via PyMem_SetAllocator. The C-API documentation names them PYMEM_DOMAIN_RAW, PYMEM_DOMAIN_MEM, and PYMEM_DOMAIN_OBJ (C-API: Memory Management). The cardinal rule the docs state up front is domain symmetry: “The APIs used to allocate and free a block of memory must be from the same domain” — memory taken with PyMem_Malloc must be released with PyMem_Free, never PyObject_Free or libc free(). Mixing domains is undefined behavior with, as the docs warn, “fatal consequences.”

The raw domain (PyMem_RawMalloc, PyMem_RawCalloc, PyMem_RawRealloc, PyMem_RawFree) is the floor. It calls the system allocator directly and, crucially, is the only domain that is safe to call without an attached thread state and without holding the GIL (C-API: Memory Management). Code that runs before the interpreter is fully up, or in contexts where no PyThreadState is current, must use the raw domain. By default it maps straight onto libc malloc/free and never touches pymalloc.

The mem domain (PyMem_Malloc and friends) is for general-purpose buffers — scratch I/O buffers, temporary C arrays, anything that is not itself a PyObject. It requires an attached thread state and, in the default build, is backed by pymalloc. The canonical pattern the docs show is allocating a BUFSIZ scratch buffer for an I/O read, copying the result into a Python object, then PyMem_Free-ing the buffer.

The object domain (PyObject_Malloc, PyObject_Calloc, PyObject_Realloc, PyObject_Free) is where the bytes for actual Python objects come from — the memory a type’s tp_alloc slot hands out for a new instance. It too requires a thread state and defaults to pymalloc. This is the domain through which the overwhelming majority of a running program’s allocations flow, because in CPython [[PyObject and the Object Header|everything is a PyObject]].

The default backing, per the C-API documentation’s configuration table, is: raw → malloc; mem → pymalloc; object → pymalloc (with debug-hook wrappers in a debug build). The free-threaded build (Free-Threaded CPython, PEP 703) changes this materially: all three domains default to mimalloc, Microsoft’s thread-friendly allocator, because pymalloc’s pool design assumes the GIL serializes access (C-API: Memory Management; PEP 703). The free-threaded build also hardens the domain rule from best-practice to hard requirement: “only Python objects are allocated using the ‘object’ domain and … all Python objects are allocated using that domain,” because the cyclic GC in that build finds objects by walking mimalloc’s object-domain heaps rather than a separate linked list (ibid.).

pymalloc and the System Allocator — Why Two Layers

Underneath the mem and object domains sits pymalloc. The reason it exists is that CPython allocates an enormous number of small, short-lived objects — a single expression can create and discard dozens of temporary integers, tuples, and frames. Calling libc malloc/free for each is slow (every call may lock the system heap and walk free lists) and fragments the heap. pymalloc amortizes this by requesting memory from the system in large chunks called arenas, slicing each arena into pools (one page each), and slicing each pool into fixed-size blocks that it hands out for small requests (Objects/obmalloc.c, 3.14.0). A request that fits under pymalloc’s small-request threshold is satisfied from a pool’s free block in a few pointer operations; a request that exceeds it (pymalloc_alloc returns NULL “if nbytes > SMALL_REQUEST_THRESHOLD,” ibid.) falls through to the system allocator. The exact threshold, the arena/pool/block sizes, the size-class buckets, and the free-list management are the subject of The pymalloc Allocator and Arenas Pools and Blocks — this note deliberately stops at “small goes to pymalloc, large falls through to malloc.”

Resolved (2026-06-01)

SMALL_REQUEST_THRESHOLD is 512 bytes, verified at the v3.14.5 tag in Include/internal/pycore_obmalloc.h (#define SMALL_REQUEST_THRESHOLD 512, with NB_SMALL_SIZE_CLASSES derived as SMALL_REQUEST_THRESHOLD / ALIGNMENT). The guard pymalloc_alloc returns NULL for nbytes > SMALL_REQUEST_THRESHOLD, so requests above 512 bytes fall through to the system allocator. Full arena/pool/block detail in The pymalloc Allocator.

A second small-object optimization layers on top of even the object domain for a few hot types: free lists. Rather than return a freed small tuple or frame all the way to pymalloc, CPython keeps a short per-type cache of recently-freed instances and reuses them on the next allocation, skipping both pymalloc and the object’s re-initialization cost. Free lists are an orthogonal cache in front of the allocator stack, covered in Free Lists.

Reclamation — Reference Counting Is Primary, the Cyclic GC Is a Backstop

The second axis is when memory returns, and here the division of labor is sharp and worth stating precisely, because it is widely misdescribed. CPython’s own internals documentation is unambiguous: “The main garbage collection algorithm used by CPython is reference counting” (InternalDocs/garbage_collector.md, 3.14). Every object carries a count of live strong references in its header; [[Reference Counting Mechanics|Py_DECREF]] decrements it, and the moment it reaches zero the object is destroyed in that very call — _Py_Dealloc runs the type’s tp_dealloc, which releases the object’s resources, decrements the objects it referenced (potentially cascading), and returns its bytes to the allocator domain it came from (Objects/object.c, 3.14.0). This is immediate and deterministic: there is no separate sweep deciding when to free an acyclic object. The full mechanics — the macro bodies, the dealloc path, the trashcan, immortal short-circuiting — are Reference Counting Mechanics; the conceptual model and its consequences are Reference Counting.

Reference counting has exactly one structural blind spot, and it is the entire reason a second mechanism exists. The internals doc states it directly: “The main problem with the reference counting scheme is that it does not handle reference cycles.” If object A references B and B references A, then dropping every external reference still leaves each with a count of one (held by the other), so neither ever reaches zero, and refcounting never frees them — the memory leaks. “For this reason some additional machinery is needed to clean these reference cycles between objects once they become unreachable” (ibid.). That machinery is the cyclic garbage collector: a generational, cycle-detecting collector that periodically finds groups of container objects reachable only from each other and reclaims them. It is explicitly a backstop, not the primary system — it “only focuses on cleaning container objects (that is, objects that can contain a reference to one or more objects)” such as lists, dicts, and class instances (ibid.), because only containers can form cycles. A str or an int can never reference another object, so the GC never tracks it; such objects are always reclaimed by refcounting alone.

The two mechanisms are complementary, not redundant. Refcounting handles the overwhelming common case — acyclic garbage — immediately, cheaply, and deterministically. The cyclic GC handles the rare residue refcounting leaves behind, on its own (tunable) schedule. The generational design, the trial-deletion cycle-detection algorithm, finalization (__del__) ordering, weak references, and the cautionary tale of the incremental collector reverted in 3.14.5 are all the subject of §7 of the Python Internals MOC — The Cyclic Garbage Collector, Generational Garbage Collection, and Reference Cycles and How They Form. This note’s job is only to place the GC correctly in the picture: below refcounting in priority, orthogonal to the allocator stack, and scoped only to cycles.

Putting It Together — The Life of an Allocation

Walking one object end-to-end ties the two axes together. When Python evaluates x = [1, 2, 3], the list type’s tp_alloc calls PyObject_Malloc (object domain). pymalloc finds a free block in a pool sized for the list header and hands it back; the list’s ob_refcnt is initialized to 1 (Include/object.h, 3.14.0). As the name x and any other references come and go, Py_INCREF/Py_DECREF adjust the count (see Reference Counting Mechanics). When the last reference drops, Py_DECREF drives the count to zero and calls _Py_Dealloc; the list’s tp_dealloc decrefs its three elements and then frees the list’s own bytes back to pymalloc’s pool — which may keep them for the next allocation or, if a whole arena empties, return the arena to the system malloc. The cyclic GC is never consulted unless the list had been made part of a cycle (e.g. x.append(x)); only then, when refcounting leaves it stranded, does the generational collector eventually find and reclaim it.

Common Misunderstandings

  • “Python’s garbage collector frees my objects.” Mostly false. The gc module’s collector reclaims cycles; the routine reclamation of virtually all objects is refcounting, which is not “the GC” in the gc-module sense. Calling gc.disable() does not stop memory from being freed — acyclic objects still die at refcount zero. It only disables cycle collection (and risks leaking cyclic garbage). This is the single most common confusion the layered model corrects.

  • “pymalloc replaces malloc.” No — it sits in front of malloc for small object/mem-domain requests and falls through to malloc for large ones and for arena refills. The raw domain bypasses pymalloc entirely. The system allocator is always the floor.

  • “All three domains are interchangeable.” No — the domain-symmetry rule (free with the matching domain’s free) is a hard correctness requirement, made enforced (not merely advised) in the free-threaded build. Freeing object-domain memory with libc free() corrupts the heap (C-API: Memory Management).

  • “Refcount reaching zero means the GC runs.” No — reaching zero invokes tp_dealloc synchronously, in the decref. There is no deferral and no collector involvement for the acyclic case.

  • “Immortal objects are freed eventually.” No — immortal objects (None, True, small ints, static types; PEP 683, since 3.12) have their refcount pinned so high that Py_INCREF/Py_DECREF are no-ops on them; they are never reclaimed except at interpreter teardown. They short-circuit the reclamation axis entirely. Detail in Reference Counting Mechanics and Immortal Objects.

Configurability and Diagnostics

The whole allocator stack is swappable at runtime. PyMem_GetAllocator/PyMem_SetAllocator let an embedder replace any domain’s function table, and the PYTHONMALLOC environment variable selects the configuration: PYTHONMALLOC=malloc disables pymalloc (routing the mem/object domains to libc malloc for debugging with tools like Valgrind), and PYTHONMALLOC=debug installs debug hooks that pad allocations with guard bytes to catch buffer overruns and use-after-free (C-API: Memory Management). For measuring rather than swapping, [[tracemalloc and Memory Profiling|tracemalloc]] hooks the domains to attribute every live allocation back to the Python source line that made it. These are the practical entry points into the system this note maps; their details live in The Memory Allocator Domains and tracemalloc and Memory Profiling.

See Also

Graph View

Backlinks