Subinterpreters

A subinterpreter is a second (third, Nth) independent copy of the Python runtime living inside a single operating-system process. Each interpreter has its own __main__ module, its own sys.modules import cache, its own set of loaded modules, classes, functions, and variables — and, since PEP 684 (Python 3.12), its own Global Interpreter Lock (GIL). Because the GILs are independent, two subinterpreters running on two OS threads can execute Python bytecode truly in parallel on two cores, all without leaving the process. CPython has been able to do this through the C API for over twenty years, but Python 3.14 exposes it to pure-Python code for the first time via the new concurrent.interpreters module (PEP 734) (per What’s New in 3.14, PEP 734).

Why Subinterpreters — Parallelism Without Leaving the Process

The motivation is the same constraint that drives multiprocessing: the GIL lets only one thread run Python bytecode at a time within one interpreter, so threads cannot give CPU parallelism for pure-Python work (see The Global Interpreter Lock and Python Threading Model). Historically the only in-language escape was to spawn whole processes. Subinterpreters offer a middle path: instead of N processes each paying full OS-process startup and memory cost, you run N interpreters inside one process, each with its own GIL, so they run in parallel — but they share the process’s file descriptors, its loaded shared libraries, and (for a narrow, carefully controlled set of objects) some memory.

PEP 734’s own framing: “The goal is to make the existing multiple-interpreters feature of CPython more easily accessible to Python code. This is particularly relevant now that CPython has a per-interpreter GIL and people are more interested in using multiple interpreters” (PEP 734). The per-interpreter GIL is the piece that turns a long-dormant curiosity into a parallelism tool — before PEP 684, all subinterpreters in a process shared one GIL, so they gave isolation but no parallelism.

Two Prerequisites — PEP 684 and PEP 683

Subinterpreter parallelism rests on two pieces of runtime surgery that landed earlier and are worth understanding because they explain the model’s limitations.

PEP 684 — A Per-Interpreter GIL (Python 3.12). For each interpreter to have its own lock, almost all mutable runtime state had to move out of process-global C variables and into each interpreter’s PyInterpreterState structure — “the GIL” itself among them (PEP 684). A new C-API entry point, Py_NewInterpreterFromConfig(), takes a PyInterpreterConfig with an own_gil field; when true, the new interpreter gets its own lock and can run on its own core concurrently with others. This is what concurrent.interpreters.create() uses under the hood.

PEP 683 — Immortal Objects (Python 3.12). Here is the subtlety: even truly immutable singletons like None, True, and False carry a reference count, and incrementing/decrementing a refcount is a write. If two interpreters with separate GILs shared one None object, they would race on its refcount with no lock protecting it. PEP 683’s answer is to make such objects immortal — their refcount is pinned to a sentinel value and Py_INCREF/Py_DECREF become no-ops, so there is no write, hence no race. As PEP 683 puts it: “Without a shared GIL, two running interpreters could not safely share any objects, even otherwise immutable ones like None… With immortal objects, support for a per-interpreter GIL becomes much simpler” (PEP 683). (See Immortal Objects for the full mechanism, including the 3.14 caveat that interned strings are mortal — only static singletons are immortal.) This is precisely why interpreters “never share objects (except in very specific cases with immortal, immutable builtin objects)” (PEP 734).

Mental Model

flowchart TB
    subgraph proc["ONE OS process"]
        direction TB
        MI["Main interpreter<br/>own GIL · own sys.modules · own __main__"]
        SI1["Subinterpreter A<br/>own GIL · own sys.modules · own __main__"]
        SI2["Subinterpreter B<br/>own GIL · own sys.modules · own __main__"]
        Q(["Queue<br/>(shareable, in-process)"])
        MI -. put/get .-> Q
        SI1 -. put/get .-> Q
        SI2 -. put/get .-> Q
        IMM["Immortal singletons<br/>(None/True/False/static types)<br/>safely shared, never refcounted"]
        MI --- IMM
        SI1 --- IMM
        SI2 --- IMM
    end
    SI1 ==> CORE1[(Core 1)]
    SI2 ==> CORE2[(Core 2)]

What it shows: three interpreters inside one process, each with its own GIL and its own module/namespace state, communicating through an in-process Queue and sharing only immortal builtin singletons. The insight: the heavy double-arrows to separate cores are the payoff — independent GILs mean A and B run Python bytecode genuinely in parallel — but everything else (modules, classes, globals) is duplicated per interpreter, not shared, which is both the safety guarantee and the memory cost.

The concurrent.interpreters API

PEP 734 was “accepted with the provision that the name change to concurrent.interpreters” (PEP 734), and the shipped 3.14 public module is indeed concurrent.interpreters — confirmed against both the 3.14 What’s New (“the new concurrent.interpreters module”) and the concurrent.interpreters docs page. This is the first time the feature is available to pure-Python code; PEP 554 was the earlier, deferred version of the same proposal (whose draft module was bare interpreters), so the two are easy to conflate — but as of 3.14 the answer is unambiguous. The module-level functions are create(), get_current(), get_main(), list_all(), and create_queue() (docs).

API name discipline

The shipped 3.14 Interpreter API has exec(), call(), and call_in_thread() — there is no run() method (some pre-release drafts and tutorials show interpreters.run(); that is not the 3.14 API). Verified against the concurrent.interpreters docs.

Creating and Running Code

from concurrent import interpreters
 
interp = interpreters.create()        # new isolated interpreter, its own GIL
 
interp.exec('print("spam!")')         # run SOURCE STRING in interp's __main__,
                                      #   on the CURRENT OS thread (serial!)
 
def run(arg):
    return arg
 
res = interp.call(run, 'spam!')       # call a CALLABLE in interp, current thread
print(res)                            # 'spam!'
 
t = interp.call_in_thread(run, 'eggs')# run in interp on a NEW OS thread →
t.join()                              # this is where parallelism actually happens

Walking it: interpreters.create() builds a fresh interpreter (with own_gil=True) and returns an Interpreter object — a handle, not the interpreter itself. interp.exec(code) “runs source code in the interpreter” by executing the string in that interpreter’s __main__ module (docs); interp.call(callable, *args) invokes a Python callable inside the interpreter and returns its result. The critical, easy-to-miss detail: exec() and call() run on the calling thread — they block, and they do not give you parallelism. To get two interpreters running on two cores at once you must run them on different OS threads, which is exactly what interp.call_in_thread(callable, *args) does (or you spin up your own threading.Threads, each driving a different interpreter). interp.close() finalizes and destroys the interpreter; interp.is_running() reports whether it is executing in __main__; interp.id and interp.whence are read-only metadata.

Passing Data In — prepare_main

prepare_main(ns=None, **kwargs) binds objects into the target interpreter’s __main__ namespace before you run code there (docs). “For most objects a copy will be bound in the interpreter, with pickle used in between” (PEP 734):

tasks  = interpreters.create_queue()
results = interpreters.create_queue()
 
interp.prepare_main(tasks=tasks, results=results)   # bind two queues into __main__
interp.exec("""if True:
    from mymodule import handle_request
    while True:
        req = tasks.get()       # blocks until parent puts work
        if req is None:
            break
        results.put(handle_request(req))
""")

prepare_main(tasks=..., results=...) makes the names tasks and results available as module globals inside the subinterpreter’s __main__, so the code string can reference them directly. Note the worker imports handle_request itself — each interpreter has its own sys.modules, so a module imported in the parent is not visible in the child; the child re-imports it into its own namespace.

Queues — Passing Data Between Interpreters

Because interpreters do not share object graphs, you move data between them with an interpreters.Queue (from create_queue()), which implements the familiar queue.Queue interface — put(), get(), put_nowait(), get_nowait() (docs):

tasks = interpreters.create_queue()
tasks.put(req)                  # serialize/share req into the queue
res = results.get(timeout=0.1)  # pull a result back

A Queue is itself shareable, which is why you can prepare_main(tasks=tasks) it into a child. Two interpreter-specific exceptions wrap the standard ones: QueueEmptyError (a subclass of queue.Empty, raised by get/get_nowait) and QueueFullError (a subclass of queue.Full).

What Can Cross the Boundary — Shareable Objects

This is the central restriction and the place the model differs most from plain function calls. Objects that can cross between interpreters fall into three groups (docs):

• Passed efficiently (copied or shared without full pickling): None, True, False, bytes, str, int, float, and tuples of such objects.
• Genuinely share their underlying mutable data: memoryview (the buffer is actually shared, not copied — the one window for zero-copy data exchange) and interpreters.Queue.
• Everything else: copied via pickle — “All objects that can be pickled are shareable. Thus, nearly every object is shareable” (PEP 734).

If an object can neither be shared nor pickled, attempting to send it raises NotShareableError (a subclass of TypeError) (docs). The mental model: passing data to a subinterpreter is more like multiprocessing (copy across a boundary) than like a thread (shared heap) — with the notable exception that memoryview buffers can be shared in-process, which multiprocessing can only achieve via the separate shared_memory mechanism.

Errors From the Other Side

When code run via exec() or call() raises an uncaught exception, it surfaces in the caller as ExecutionFailed (a subclass of InterpreterError), carrying an excinfo snapshot of the original exception — type, message, and a traceback snapshot (docs, PEP 734). The original exception object itself cannot simply be handed across (it lives in the other interpreter), so you get a serialized representation. Other exceptions: InterpreterError (base) and InterpreterNotFoundError (the target interpreter no longer exists).

InterpreterPoolExecutor — The High-Level Wrapper

For pool-style work you do not have to drive interpreters and threads by hand. Python 3.14 also adds concurrent.futures.InterpreterPoolExecutor, “a new executor class… which exposes multiple Python interpreters in the same process (‘subinterpreters’) to Python code. This uses a pool of independent Python interpreters to execute calls asynchronously” (What’s New in 3.14). It has the same submit()/map() API as the rest of The concurrent.futures Framework, so swapping a ProcessPoolExecutor for an InterpreterPoolExecutor is often a one-line change — but the same shareability rules apply (arguments and return values must cross the interpreter boundary). It is “separate from the new interpreters module” but built on the same machinery.

Failure Modes and Common Misunderstandings

“exec() runs it in parallel.” No. exec() and call() run on the calling thread and block. Parallelism requires call_in_thread() or your own threads, one per interpreter. This is the single most common misconception.

Passing an unshareable object. Sending a non-picklable, non-shareable object (an open socket, a lambda, a thread lock) raises NotShareableError. Like multiprocessing, you pass plain data, not live resources.

C-extension incompatibility. A C extension must explicitly declare support for being loaded into a subinterpreter. The mechanism (added in 3.12) is the Py_mod_multiple_interpreters slot in a module’s multi-phase-initialization PyModuleDef_Slot array, which “determines whether or not importing this module in a subinterpreter will fail” (C-API module docs). It takes one of three values: Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED (“does not support being imported in subinterpreters”), Py_MOD_MULTIPLE_INTERPRETERS_SUPPORTED (“only when they share the main interpreter’s GIL”), or Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (“even when they have their own GIL”) (docs). The last value is the one that matters for parallel subinterpreters — an extension that wants to run under per-interpreter GILs must opt in to it explicitly. A single-phase-init extension (the classic PyInit_* returning a ready module) cannot declare this and is rejected from per-GIL subinterpreters; many widely-used C extensions have not yet migrated to multi-phase init with this slot, which is the biggest practical blocker today and the main reason subinterpreters are not yet a drop-in replacement for multiprocessing.

Uncertain (dated 2026-06-01)

Verify: which major C extensions (NumPy, lxml, the common database drivers) declare Py_MOD_PER_INTERPRETER_GIL_SUPPORTED as of mid-2026. The slot mechanism is verified against the v3.14.5 C-API docs; what is uncertain is per-library adoption, which is external to CPython and moves continuously. To resolve, check each project at the version in use, in order of authority: (1) grep the project’s C sources for Py_mod_multiple_interpreters / Py_MOD_PER_INTERPRETER_GIL_SUPPORTED in its PyModuleDef_Slot array; (2) the project’s own “free-threading / subinterpreter support” doc page or release notes; (3) attempt import inside a concurrent.interpreters interpreter created with an own GIL and observe whether it raises ImportError. As of this writing the broad signal is that few heavy native extensions have opted in, which is the primary practical blocker — but treat any specific library’s status as unverified until checked by method (1) or (3). #uncertain

Process-wide crashes. Isolation is in-process. A segfault in one subinterpreter (e.g. from a buggy C extension) takes down the whole process — unlike multiprocessing, where a crashing worker leaves the parent standing. Subinterpreters trade multiprocessing’s hard fault-isolation for lower overhead.

Alternatives and When to Choose Them

The four CPython concurrency models and where subinterpreters sit among them:

• Threads (Python Threading Model): share one heap and one GIL → trivial data sharing, no CPU parallelism for pure Python. Best for I/O-bound work.
• asyncio (The asyncio Event Loop): one thread, cooperative scheduling, massive I/O concurrency, no parallelism.
• multiprocessing: separate OS processes → parallelism and the strongest fault isolation, but high startup/memory cost, the pickle/IPC tax on everything, and the 3.14 fork→forkserver safety story. The mature, battle-tested choice.
• Subinterpreters (this note): parallelism with lower startup cost and lower memory than processes, can share memoryview buffers and pass via in-process queues, and stay inside one process — but isolation is in-process (a hard crash kills everything), and C-extension support is immature. The lighter-weight, newer bet.

In one line: subinterpreters = separate interpreters in one process (in-process isolation, own GIL → parallel, can share buffers, cheaper, immature); multiprocessing = separate OS processes (hard isolation, copy everything, always parallel, expensive, mature). Choose subinterpreters when you want process-like parallelism without process-like cost and your workload is pure Python (or uses only subinterpreter-safe extensions); choose multiprocessing when you need rock-solid fault isolation or depend on C extensions that have not yet opted in.

Production Notes

Uncertain (dated 2026-06-01)

Verify: real-world production usage of concurrent.interpreters. PEP 734’s public Python API is new in 3.14 (released October 2025), so roughly half a year into release there is not yet a meaningful body of production deployments or incident post-mortems to cite, and inventing one would violate the vault’s accuracy contract. To resolve, look for: (1) a named OSS project that adopts concurrent.interpreters or concurrent.futures.InterpreterPoolExecutor on its main/release branch (grep its repo for concurrent.interpreters); (2) a conference talk or engineering blog post reporting a measured production workload migrated from threads/multiprocessing to subinterpreters; (3) a CPython tracker issue documenting a real-world bug or post-mortem under load. Until one of those exists, treat this as an early-adopter feature, not a battle-tested one. #uncertain

What can be said with confidence: the underlying C-API subinterpreter support has existed and been used (e.g. by mod_wsgi) for two decades; PEP 734 makes it accessible to Python code without writing C. The realistic near-term adopters are CPU-parallel data-processing pipelines and the InterpreterPoolExecutor as a lighter-weight ProcessPoolExecutor substitute, pending broader C-extension compatibility.

See Also

• Per-Interpreter GIL — PEP 684; the mechanism (own_gil) that makes subinterpreter parallelism real
• multiprocessing and Process-Based Parallelism — sibling “parallelism by isolation” model: separate processes vs separate interpreters here
• The Global Interpreter Lock — the constraint both models route around
• Immortal Objects — PEP 683; why None/True/False can be shared across interpreters safely
• The concurrent.futures Framework — InterpreterPoolExecutor is the high-level pool over this
• The pickle Protocol — how non-shareable objects cross the interpreter boundary
• Python Threading Model — the thread sibling; concurrency without parallelism under the GIL
• Python Internals MOC — §9 Concurrency and Parallelism