vault

The CPython Source Tree Layout

13 min read

The CPython Source Tree Layout

CPython — the reference implementation almost everyone means when they say “Python” — is a large C program (with a substantial pure-Python standard library bolted on) whose git repository is carved into a stable set of top-level directories. Each directory has one job: Python/ holds the runtime and compiler, Objects/ holds one C file per built-in type, Include/ holds the public and private C headers, Modules/ holds C extension modules, Parser/ and Grammar/ hold the front end that turns text into a parse tree, and Lib/ holds the parts of the standard library written in Python itself. Learning this map is the prerequisite for reading any deeper internals note, because every later claim — “dict is implemented in dictobject.c,” “the eval loop is in ceval.c,” “the PEG grammar lives in python.gram” — is a pointer into this tree. This note describes the layout as of the 3.14 branch (CPython 3.14.0 released 7 October 2025), verified against the live GitHub listing rather than memory.

Mental Model: One Directory, One Responsibility

The cleanest way to hold the tree in your head is to think of CPython as four concentric layers, each with its own directory neighborhood. At the center is the language front end (Grammar/, Parser/) that reads .py text and produces a syntax tree. Wrapped around it is the compiler and runtime core (Python/) that turns that tree into bytecode and then executes the bytecode. Surrounding that is the object system (Objects/) that defines what the values flowing through the interpreter actually are. And at the outside sit the batteries — the C extension modules (Modules/) and the pure-Python standard library (Lib/) — plus the interface layer (Include/) that lets all of these, and third-party extensions, talk to the core. The Programs/ directory is the thin shell that wires a main() onto the whole thing to produce the python executable.

graph TD
    subgraph FrontEnd["Front end — text to tree"]
        G["Grammar/<br/>python.gram, Tokens"]
        P["Parser/<br/>pegen.c, parser.c, tokenizer/"]
    end
    subgraph Core["Core — compile and run"]
        PY["Python/<br/>compile.c, ceval.c,<br/>bytecodes.c, gc.c, import.c"]
    end
    subgraph Objects["Object system — what values are"]
        OBJ["Objects/<br/>longobject.c, dictobject.c,<br/>unicodeobject.c, typeobject.c ..."]
    end
    subgraph Batteries["Batteries"]
        MOD["Modules/<br/>C extensions + main.c"]
        LIB["Lib/<br/>pure-Python stdlib + Lib/test/"]
    end
    INC["Include/<br/>public + cpython/ + internal/ headers"]
    PROG["Programs/python.c<br/>(266-byte wrapper)"]

    G --> P --> PY
    PY <--> OBJ
    PY --> MOD
    PY --> LIB
    INC -. "consumed by" .- PY
    INC -. "consumed by" .- OBJ
    INC -. "consumed by" .- MOD
    PROG --> MOD
    PROG --> PY

    style FrontEnd fill:#e8f0ff
    style Core fill:#fff0e8
    style Objects fill:#e8ffe8
    style Batteries fill:#fff8e0

Figure: the CPython source tree grouped by responsibility. The insight to extract is the flow left-to-right and the special role of Include/ and Programs/: headers in Include/ are the contract every layer compiles against (and the foundation of the C API ecosystem), while Programs/python.c is a near-empty stub — the real startup logic lives in Modules/main.c. Arrows show “uses”; the dashed lines show “compiles against these headers.”

The Directories, One by One

The authoritative description of this layout is the CPython Developer’s Guide, which devotes a “lay of the land” table to it (devguide: setup and building). What follows expands each entry with the specific files that recur in the rest of the internals notes, all confirmed against the GitHub 3.14 branch listing.

Python/ — the runtime, compiler, and eval loop

Python/ is the beating heart. Despite the name it does not hold any .py files; it holds the C source for the compiler back end and the interpreter runtime. The single most important file is ceval.c — roughly 116 KB of C (116,643 bytes; confirmed at the v3.14.5 tag) — which contains the evaluation loop _PyEval_EvalFrameDefault that fetches and executes one bytecode instruction at a time. Alongside it sit compile.c (turns the abstract syntax tree into a code object full of bytecode), symtable.c (builds the symbol tables that classify every name as local, global, or free), import.c (the C core of the import system), gc.c (the cyclic garbage collector), bltinmodule.c (the builtins module — print, len, range, and friends), pythonrun.c (the high-level “run this string/file” entry points), and sysmodule.c (the sys module).

A 3.11-and-later wrinkle that is worth internalizing: the interpreter’s instruction definitions are no longer hand-written as a giant switch in ceval.c. They live in Python/bytecodes.c, a domain-specific-language (DSL) source file (~225 KB on 3.14). A build-time generator reads bytecodes.c and emits Python/generated_cases.c.h (the conventional interpreter cases, ~528 KB) and Python/executor_cases.c.h (the tier-2 micro-op cases consumed by the JIT). This is why grepping ceval.c for a specific opcode often fails — the opcode body is generated from the DSL. See CPython Compilation Pipeline and The Specializing Adaptive Interpreter for what that machinery produces and consumes.

Objects/ — one C file per built-in type

Objects/ is where the convention worth memorizing lives, because it recurs across §4 (the object model) and §5 (built-in type internals) of the Python Internals MOC. Each built-in type gets its own .c file, named <type>object.c. The mapping is mechanical and was confirmed against the 3.14 Objects/ listing:

  • longobject.c → the arbitrary-precision int (CPython calls it “long” for historical reasons — see CPython Integer Internals).
  • dictobject.c → the compact, insertion-ordered dict (CPython Dict Internals).
  • unicodeobject.c → str, with its three storage widths (CPython String Internals).
  • listobject.c, tupleobject.c, setobject.c, floatobject.c, bytesobject.c, bytearrayobject.c, complexobject.c, rangeobject.c, boolobject.c → the rest of the familiar built-ins.

Two files in Objects/ are not a single type but the machinery shared by all of them. object.c holds the generic operations that work on any PyObject (the universal object header, generic attribute access, the default repr), and typeobject.c implements type itself — the PyTypeObject that every object points to and the C3 MRO computation. One more lives here for historical reasons: obmalloc.c, the pymalloc small-object allocator, even though it is a memory-management concern rather than an object type. The 3.14 tree also shows newer entries like interpolationobject.c and templateobject.c — these back the template strings (t-strings) added in 3.14 (What’s New in Python 3.14) (see Python 3.14 Release Internals), a concrete example of “new language feature ⇒ new *object.c file.”

Include/ — the C-API headers in three tiers

Include/ holds the header files that every C compilation unit — core, extension module, or third-party package — #includes. Its internal structure is itself a load-bearing design decision, because it encodes the stability tiers of the C API. The 3.14 listing shows three levels:

  1. Include/*.h (top level) — the headers reachable through #include <Python.h>. These are where the Limited API is drawn from, but be precise: the Limited API is a curated subset of the full C API that you opt into by defining the Py_LIMITED_API macro, which “hide[s] all definitions that are not part of the [stable] ABI” (PEP 384). So a top-level header contains both limited and non-limited symbols; the macro is the filter, not the directory. An extension compiled with Py_LIMITED_API against this subset keeps loading across feature releases without recompilation, because those symbols are backed by the stable ABI (The Stable ABI and Limited API). object.h, abstract.h, dictobject.h, floatobject.h and so on live here.
  2. Include/cpython/ — public but CPython-specific declarations. Python.h pulls these in by default, but they are skipped when Py_LIMITED_API is defined: they are part of the public C API yet expose implementation details, so they are excluded from the stable ABI and may change between feature releases.
  3. Include/internal/ — the private core. Headers here require the Py_BUILD_CORE macro to be defined and are not a supported interface for extensions at all; they exist so the interpreter’s own translation units can share declarations. This is where structures like the internal interpreter and thread state live.

The reason this three-way split exists is the whole point of The Stable ABI and Limited API: it lets CPython evolve its internals freely (the internal/ tier), expose a richer-but-volatile surface for tools that need it (the cpython/ tier), and still promise binary compatibility to the broad ecosystem (the top-level Limited API tier). When a header is moved down a tier — say from public to internal/ — that is a deliberate API-narrowing event. Note refcount.h, critical_section.h, lock.h, and pyatomic.h at the top level: these surfaced as the free-threaded build needed atomic refcounting and per-object locking primitives.

Modules/ — C extension modules and the real main()

Modules/ holds standard-library modules that are written in C for speed or for access to operating-system facilities, as distinct from the pure-Python modules in Lib/. The 3.14 listing includes arraymodule.c (array), mathmodule.c/cmathmodule.c (math/cmath), socketmodule.c, _asynciomodule.c (the C-accelerated asyncio core), _pickle.c (The pickle Protocol), _json.c, itertoolsmodule.c, posixmodule.c (the POSIX layer behind os), and directory-form packages like _io/, _decimal/, _sqlite/, and _ctypes/. The naming convention is “an underscore prefix marks the C-accelerator half of a module whose public face is a Python wrapper in Lib/” — e.g. Modules/_pickle.c is the fast path that Lib/pickle.py imports when available.

Crucially, Modules/ also contains main.c, and this is where the program actually starts. The function Py_BytesMain / Py_RunMain — the logic that parses command-line arguments, sets up sys.argv, configures the interpreter, and either runs a script, a module (-m), or the REPL — lives here, not in Programs/.

Programs/ — the executable stub

Programs/ is small and easy to misread. The marquee file, Programs/python.c, is only 266 bytes at the v3.14.5 tag (confirmed) — it does essentially nothing but call Py_BytesMain(argc, argv) and return its result. The substance lives in Modules/main.c (above). The mental correction here matters: when people say “the entry point is Programs/python.c,” they mean it is the file that gets compiled and linked into the python binary, but the interesting startup code is one indirection away. The directory also holds _freeze_module.c (used to freeze stdlib modules into the binary — see Frozen and Built-in Modules), _bootstrap_python.c (a minimal interpreter used during the build to run the freezing step), and _testembed.c (tests for embedding CPython in another C program).

Parser/ and Grammar/ — the front end

Since CPython 3.9, the parser is a PEG (Parsing Expression Grammar) parser (PEP 617), and the two directories split its definition from its generated code. Grammar/ holds the specification: Grammar/python.gram (~72 KB on 3.14) is the PEG grammar that defines Python’s syntax, and Grammar/Tokens lists the token types. Parser/ holds the engine and generated code: pegen.c/pegen.h are the PEG runtime, parser.c (a very large file, ~1.4 MB) is the parser generated from python.gram by a build-time tool, and action_helpers.c builds AST nodes during the parse. Parser/ also contains the tokenizer (tokenizer/ subdirectory) and lexer (lexer/ subdirectory), Parser/Python.asdl (the Abstract Syntax Description Language definition of the AST node types, from which asdl_c.py generates the AST C structs), and string_parser.c (which parses the contents of f-strings). See The PEG Parser and Python Tokenizer for how these cooperate.

Lib/, Tools/, Doc/, and the rest

Lib/ is the standard library written in Python — os.py, argparse.py, json/, asyncio/, and hundreds more. Critically, Lib/ also contains Lib/test/, CPython’s own regression test suite (run with python -m test), and Lib/importlib/, the pure-Python reimplementation of the import system that is itself frozen into the interpreter at build time. Tools/ holds maintenance utilities — the code generators that read bytecodes.c and python.gram, the clinic argument-clinic tool, release scripts, and so on; it is “scripts that build or maintain CPython, not part of the shipped runtime.” Doc/ is the reStructuredText source for docs.python.org. The remaining top-level directories are platform-specific or build-specific: PC/ and PCbuild/ (Windows source and MSVC build files), Mac/, Apple/, iOS/, Android/, and Platforms/ (the newer mobile/Apple-platform support trees), Misc/ (the NEWS changelog and developer notes), and InternalDocs/ (in-tree design documents for contributors).

Walking the Tree from a Build

The layout is easiest to internalize by tracing what make does. A from-source build runs ./configure (generated from configure.ac by autoconf, both at the repository root) to probe the platform, then make compiles the C sources in Parser/, Objects/, Python/, and Modules/ into object files, links them with the stub in Programs/python.c, and produces the python binary in the build directory. The pure-Python Lib/ is not compiled — it is found at runtime via the interpreter’s path configuration (with a frozen copy of importlib baked in so that import works before any .py file is read). The Developer’s Guide’s build instructions (devguide: setup and building) walk this end to end.

git clone https://github.com/python/cpython         # the whole tree above
cd cpython
./configure --with-pydebug                          # probe platform; --with-pydebug adds assertions
make -j$(nproc)                                      # compile Parser/ Objects/ Python/ Modules/ -> ./python
./python -c "import sys; print(sys.version)"         # the stub in Programs/python.c, calling Modules/main.c

Line by line: git clone pulls the directory tree this note describes; ./configure reads configure.ac/pyconfig.h.in and writes a platform-specific Makefile and pyconfig.h; make builds the C layers into ./python; the final command runs that binary, which starts in Programs/python.c → Modules/main.c, brings up the runtime in Python/pylifecycle.c, and imports the pure-Python Lib/ on demand.

Common Misunderstandings

“Python/ contains the .py standard library.” No — Python/ is C source for the runtime and compiler; the .py standard library is in Lib/. The naming is a perennial trip-hazard.

“Programs/python.c is where startup logic lives.” It is the linked entry file, but it is a 266-byte wrapper around Py_BytesMain; the real startup code is Modules/main.c and Python/pylifecycle.c.

“There’s a ceval.c switch I can read to find an opcode.” Since 3.11 the opcode bodies are written in the Python/bytecodes.c DSL and generated into generated_cases.c.h. Read bytecodes.c, not ceval.c, to see what an instruction does.

“Grammar/python.gram is the parser.” It is the grammar specification. The parser C code (Parser/parser.c) is generated from it. Editing python.gram requires regenerating parser.c via the Tools/ generator and rebuilding.

“Headers are headers.” The three-tier split (Include/*.h Limited API vs Include/cpython/ vs Include/internal/) is the entire stable-ABI story; treating an internal/ header as a public API is exactly the mistake that breaks an extension on the next feature release.

File sizes pinned to v3.14.5 (verified 2026-06-01)

The byte sizes quoted above are exact at the v3.14.5 release tag, confirmed via the GitHub contents API on 2026-06-01: Python/ceval.c = 116,643 bytes (≈116 KB), Parser/parser.c = 1,406,261 bytes (≈1.4 MB), Programs/python.c = 266 bytes. Sizes drift across patch releases; the existence and role of each file is stable. Re-read the v3.14.5 tree if exact sizes for another release matter.

Alternatives and Contrast

Other Python implementations lay out their source very differently, which is a useful reminder that this tree is a CPython artifact, not a property of the language (see Python Language vs CPython Implementation and CPython vs PyPy and Alternative Implementations). PyPy’s repository is organized around its RPython meta-tracing toolchain and has no Objects/-style file-per-type C layout at all; its interpreter is itself written in RPython. Compared with a statically compiled language like Go (see Go Internals MOC), the striking structural difference is that CPython ships a large interpreted standard library (Lib/) right next to the C runtime, and the front end (Parser/, Grammar/) runs at program startup on every .py file rather than once at compile time — the source tree’s split between “C core” and “Python Lib/” mirrors that runtime split.

See Also

Graph View

Backlinks