vault

The Incremental GC and Its Reversion

15 min read

The Incremental GC and Its Reversion

For four patch releases — Python 3.14.0 through 3.14.4 — CPython’s cyclic garbage collector was incremental: a redesign by Mark Shannon that collapsed the historic three generations into two (a “young” and an “old” generation) and, on each automatic collection, scanned the whole young generation but only a slice of the old generation, bounding the worst-case pause time (What’s New in Python 3.14; gh-108362). The design traded pause time for peak memory, and in production that trade went badly: large, long-lived cyclic structures (notably in the ssl/urllib3 stack) accumulated faster than the incremental old-generation slices could reclaim them, and resident set size (RSS) ballooned (gh-142516). On 10 May 2026, Python 3.14.5 shipped a forward-port of the 3.13 generational collector, reverting the incremental design entirely “due to a number of reports of significant memory pressure in production environments” (3.14.5 release notes; What’s New in Python 3.14). This note is a cautionary tale about shipping a collector rewrite without a Python Enhancement Proposal (PEP), and about how synthetic GC benchmarks can hide a memory regression that only real workloads expose.

Uncertain (dated 2026-06-01)

Several precise figures below — Neil Schemenauer’s “5x worst-case RSS”, “1.3 ms vs 26 ms max pause”, “2.7x peak RSS”, and the “~1600 lines” of the incremental compilation unit — come from the discuss.python.org reversion thread as summarized via fetch (discuss thread), not from directly quoted posts. The direction (incremental GC trades much higher peak memory for much lower pause time) is corroborated by the official What’s New and 3.14.5 release notes, and the API-revert facts are independently verified against the live gc docs’ “Changed in version 3.14.5” notes. Only the exact magnitudes remain secondary. To resolve: re-read the individual Schemenauer posts in the thread and quote the numbers directly. #uncertain

This note assumes familiarity with how CPython detects cycles at all — that logic lives in The Cyclic Garbage Collector — and with the generational scheme the revert restored, covered in Generational Garbage Collection. It focuses narrowly on what the incremental design changed, why it regressed, and what shipped in its place. The version timeline it sits inside is Python Release Cycle and Versioning.

Mental Model

Think of cyclic garbage collection as a chore that must occasionally walk the entire heap of container objects (the only objects that can form cycles) to find unreachable loops. The historic CPython collector did this in bursts: most collections touched only the youngest, smallest generation and were cheap, but a “full” collection of the oldest generation walked every surviving long-lived object at once. On a process with a large heap — say, a web server holding millions of live objects — that full collection is a single long stop-the-world pause, because nothing else can run while the collector mutates reference bookkeeping.

The incremental collector’s idea was to never do the whole chore at once. Instead of one big sweep of the old generation, it would do a little bit on every collection: scan the young generation in full (always cheap, because it is kept small) plus a bounded increment of the old generation. Over many collections those increments add up to a full pass — a “scavenge” — but no single collection stalls the program for long. The cost of that smoothing is that an object trapped in a cycle in the unscanned part of the old generation lives until the increments eventually reach it, so the heap holds more dead-but-not-yet-collected memory at any instant.

flowchart TB
    subgraph GEN["Generational GC (3.13, and 3.14.5+)"]
        direction LR
        Y0["gen 0<br/>(young)"] -->|survives| Y1["gen 1<br/>(middle)"]
        Y1 -->|survives| Y2["gen 2<br/>(old)"]
        Y2 -.->|"FULL collection:<br/>scan ALL of gen 2<br/>= one long pause"| FULL["⏱ long stop-the-world<br/>pause on big heaps"]
    end
    subgraph INC["Incremental GC (3.14.0 – 3.14.4)"]
        direction LR
        IY["young gen<br/>(scanned fully<br/>every collection)"] -->|survives| IO["old gen"]
        IO -->|"each collection scans<br/>young + a SLICE of old<br/>(pending → visited)"| SLICE["⏱ short bounded pause<br/>BUT dead cycles in<br/>unscanned old gen linger<br/>→ RSS grows"]
    end

Diagram: the two designs side by side. The generational collector (top) reclaims old-generation cycles in a single full sweep — short heaps are fine, but a large old generation makes that sweep a long pause. The incremental collector (bottom) replaces the big sweep with many small slices, flattening the pause-time spike. The insight to take away is that the incremental scheme moves the cost from the time axis (one long pause) to the memory axis (dead cycles wait longer to be freed, so peak RSS rises) — and it was that memory cost that proved unacceptable in production.

Why CPython Even Tried This

The motivation predates 3.14. In gh-100403, Pablo Galindo Salgado questioned whether the weak generational hypothesis — the assumption that most objects die young, which justifies collecting the youngest generation most often — even holds for CPython. His argument: because CPython reclaims most short-lived objects through reference counting (see The Cyclic Garbage Collector for how refcounting and cycle detection divide the labor), the objects that reach the cyclic collector are a biased sample — they have already survived refcounting, so the young generations of the cycle collector see very low success rates. He measured collection success rates running black over the standard library and found generation 0 and generation 1 collections reclaimed almost nothing, while generation 2 (the oldest) did most of the real work (gh-100403). That data suggested the three-generation structure was largely wasted overhead.

Mark Shannon’s gh-108362 took the complementary angle: “The current GC is both inefficient and can have very long pause times… We should use an incremental collector, it can improve efficiency and hugely reduce maximum pause times.” The promise in the What’s New was concrete: “maximum pause times are reduced by an order of magnitude or more for larger heaps” (What’s New in Python 3.14). For latency-sensitive servers — the canonical victim of a multi-millisecond GC stall — that was a genuine win.

How the Incremental Design Worked

The incremental collector reduced the generation count from three to two: a young generation and an old generation (What’s New in Python 3.14). Newly tracked container objects enter the young generation. Per the CPython internal design doc as of v3.14.0, “each garbage collection scans the entire young generation and part of the old generation” (InternalDocs/garbage_collector.md).

The old generation is split into two logical lists that together implement the incremental scan:

  • a pending list — old-generation objects not yet examined in the current full scavenge, and
  • a visited list — objects already examined this scavenge.

On each automatic collection, the collector processes the young generation plus the least-recently-scanned objects from the old generation’s pending list, plus all objects transitively reachable from that initial increment that have not yet been scanned this scavenge (InternalDocs/garbage_collector.md). That transitive-closure step is what keeps correctness intact: cycle detection requires that you not declare an object dead while you still hold an unexamined reference into its cluster, so the increment is expanded to cover the reachable, unscanned frontier before the increment is “closed.” Surviving objects are moved to the back of the visited list; once the pending list empties, the visited and pending roles swap and a new full scavenge begins.

The bookkeeping reuses the same gc_refs field that the non-incremental collector uses for its trial reference-count computation. As The Cyclic Garbage Collector explains, the collector copies each tracked object’s reference count into a private gc_refs slot (stored in the GC header’s _gc_prev bits) and then decrements it for every internal reference found during traversal; an object whose gc_refs reaches zero is reachable only from within the candidate set and is provisionally unreachable. The incremental scheme threads this same machinery through the pending/visited partition, so the increment is computed over only the slice of objects in scope rather than the whole old generation at once.

Two user-visible consequences fell out of the redesign, both documented in What’s New and the gc module reference:

  1. gc.collect(1) changed meaning. Under the incremental collector, “gc.collect(1): Performs an increment of garbage collection, rather than collecting generation 1” (What’s New in Python 3.14). Because there was no longer a distinct middle generation, passing 1 requested one increment of the old-generation scan instead.
  2. gc.get_objects(1) and set_threshold’s third argument lost meaning. The gc docs note that in 3.14 “Generation 1 removed” from gc.get_objects, and threshold2 was “ignored” by set_threshold (gc module docs) — both natural results of dropping to two generations. (See The gc Module for the full API surface and GC Thresholds and Tuning for the threshold semantics.)

Why It Regressed in Production

The trade-off was honest on paper — lower pause time, higher peak memory — but the memory side turned out to be far worse than a constant factor for a specific, common workload: steady creation of cyclic garbage in a long-lived process.

The clearest field report is gh-142516, titled “Observed memory leak in ssl library: Python 3.14 GC issue.” A user running the MSAL (Microsoft Authentication Library) for Python — which sits on requests → urllib3 → ssl — saw RSS grow steadily after upgrading from 3.13 to 3.14 (tested on 3.14.0, .1, .2), traced via memray to repeated ssl.SSLContext.load_verify_locations calls. The SSLContext objects formed cycles that, under the incremental collector, sat in the unscanned portion of the old generation and were reclaimed only slowly — so to an operator watching a dashboard, it looked exactly like a leak even though the memory was eventually collectable.

Neil Schemenauer’s synthetic benchmarking (his “cyclotron” tool, posted to the reversion thread on 17–19 April 2026) characterized the failure mode directly. As relayed in the thread summary: “if there is a lot of cyclic garbage being created… process memory use can be dramatically higher (5x was the worst case I saw) and runtime is slower (more time spent in GC, longer total execution time).” In one run he measured “1.3 ms max pause with incremental GC, 26 ms max pause with generational GC. Peak RSS size was 2.7x with incremental GC though” (discuss thread). The pause-time win was real and large; the memory cost was real and also large — and on a server, an out-of-memory kill is a harder failure than a 26 ms pause.

There is a second, subtler factor visible in gh-100403’s data. Because the weak generational hypothesis fits CPython poorly, most of what the incremental collector defers is genuine long-lived garbage in the old generation — precisely the category that the incremental design is slowest to reclaim. The scheme is best where most garbage is young (refcounting already handles that) and worst where garbage clusters in the old generation, which is where CPython’s surviving cyclic garbage actually lives.

The Decision to Revert

On 16 April 2026, Hugo van Kemenade opened the discuss.python.org thread “Reverting the incremental GC in Python 3.14 and 3.15,” stating: “Python 3.14 shipped with a new incremental garbage collector. However, we’ve had a number of reports of significant memory pressure in production environments… We’ve decided to revert it in both 3.14 and 3.15, and go back to the generational GC from 3.13” (discuss thread).

The rationale was as much about process as about benchmarks. The incremental collector had been merged without going through the PEP review process; it had, in fact, already been rolled back once before — just ahead of the 3.13 final release, after a Sphinx-documentation-build slowdown surfaced during the Bellevue development sprint — before being re-merged for 3.14 (discuss thread). With production memory regressions now confirmed, the conservative call was to return to the 3.13 generational collector, a “known quantity,” rather than to keep patching the incremental one in stable releases.

Resolved (2026-06-01) — partial

Two distinct rollback events are visible in the git history, and the “before 3.13 final” one is now pinned. (1) During early development the collector was merged (GH-108038, 2024-02-05), reverted two days later (GH-115132, 2024-02-07), then re-merged (GH-116206, 2024-03-20) — before the 3.13 branch was cut, so 3.13.0b1 shipped the incremental collector (its pycore_runtime_init.h uses the .young = { .threshold = 2000 } / .old = {…} form). (2) The “just ahead of 3.13 final” rollback is a separate commit on the 3.13 branch: GH-124567 “Revert the Incremental GC in 3.13” (2024-09-30), which restored the three-generation .generations = {{2000},{10},{10}} form seen in v3.13.0 final. The collector was then re-introduced for 3.14. The bare reason for (2) (“Sphinx-build slowdown at the Bellevue sprint”) is not in the revert commit message and remains from the reversion-thread summary; the revert sequence and its placement (b1 incremental → final generational) are primary-source verified by diffing pycore_runtime_init.h at the v3.13.0b1 and v3.13.0 tags.

Neil Schemenauer proposed an alternative — keep both collectors and let the user pick at interpreter startup (his prototype “adds about 1600 lines of code… quite well separated”), and he was “+1 to have two versions in 3.16+.” But the consensus for the stable branches favored the clean revert; a startup-selectable dual collector, if it returns, must come through a proper PEP for 3.16 or later (discuss thread).

What 3.14.5 and 3.15 Ship Instead

The mechanical revert is tracked in gh-142516: “Forward-port the generational cycle garbage collector to the default 3.14 build, replacing the incremental collector while leaving the free-threaded collector unchanged.” Note the scope: only the default (GIL-enabled) build reverts; the free-threaded build keeps its own collector implementation, which was never the incremental one.

Python 3.14.5 was released on 10 May 2026 with this text in What’s New: “The garbage collector (GC) has changed in Python 3.14.5. Python 3.14.0-3.14.4 shipped with a new incremental GC. However, due to a number of reports of significant memory pressure in production environments, it has been reverted back to the generational GC from 3.13. This is the GC now used in Python 3.14.5 and later” (What’s New in Python 3.14; 3.14.5 release notes).

Resolved (2026-06-01)

The 10 May 2026 release date is taken from the official python.org 3.14.5 release page and is authoritative. The April reversion thread’s reference to releasing “early” (ahead of an originally scheduled 9 June 2026 date) is consistent with this — the release was pulled forward, which is why the schedule discussion (April) predates the actual release (10 May). No conflict remains.

With the revert, the API changes of 3.14 were themselves undone, and the gc module reference records each reversal precisely:

  • gc.collect(1): “Changed in version 3.14.5: generation=1 performs collection of the middle generation” — i.e., the pre-3.14 meaning is back (gc module docs).
  • gc.get_objects(1): “Changed in version 3.14.5: Generation 1 reintroduced to maintain 3.13 GC behavior.”
  • gc.set_threshold: “Changed in version 3.14.5: threshold2 restored to match Python 3.13 behavior.”

In other words, a program written against the 3.13 gc API works unchanged on 3.14.5, while a program that relied on the 3.14.0–3.14.4 incremental semantics (e.g., expecting gc.collect(1) to do an increment) sees its behavior change back. Anything fast-moving here should carry its as-of date: as of 30 May 2026, the generational collector is the shipping default on 3.14.5+ and is slated for 3.15; reintroduction of an incremental scheme, if it happens, is a 3.16+ PEP question.

Lessons and Common Misunderstandings

  • “Incremental GC is the current CPython collector.” This is the single most common stale belief — and the reason this note insists on primary sources. It was current for exactly four patch releases (3.14.0–3.14.4) and is not current as of 3.14.5. Memory and older write-ups will confidently assert it is the default; verify against the live What’s New and gc docs.
  • “It was a leak.” The ssl/urllib3 symptom looked like a leak but was deferred collection of genuinely cyclic, eventually-collectable objects. The distinction matters: a true leak is never freed; the incremental collector’s problem was latency of reclamation, which under steady cyclic-garbage pressure is indistinguishable from a leak on an RSS graph until the process is OOM-killed.
  • Lower pause time is not free. The headline 10×+ pause-time reduction was achieved; it was paid for in peak RSS, and on memory-constrained production hosts that bill came due first. A GC change is a multi-dimensional trade-off, and a benchmark suite that only measures throughput and pause time can pass while the memory dimension fails.
  • Rewrites belong in PEPs. A core reason cited for the revert was that a collector rewrite of this consequence had bypassed the PEP process. The governance lesson is explicit in the thread: a future incremental collector must be PEP-reviewed and likely opt-in.

See Also

Graph View

Backlinks