vault

mcache mcentral and mheap

14 min read

mcache mcentral and mheap

Go’s small-object allocator is a three-level cache hierarchy, modelled on TCMalloc. The bottom level, mcache, is a per-P (per logical processor) cache of partially-full spans; allocating from it requires no locks at all because each P owns its own mcache exclusively. When an mcache’s span for a size class fills up, the middle level, mcentral, hands it a fresh span — there is one mcentral per span class, and accessing it takes a lock, but obtaining a whole span at once amortizes that lock over hundreds of future allocations. When an mcentral runs dry, the top level, the global mheap, carves a new span out of raw pages. The design makes the common case — a small allocation that hits the per-P cache — lock-free, while pushing the expensive, lock-taking, OS-touching operations onto a rare slow path (malloc.go top comment).

This note traces the three tiers and the flow of spans between them. The size classes and the mspan structure those spans use are covered in Size Classes and Span Management; the radix-tree page allocator the mheap sits on top of is Page Allocator; the OS reservation/commit machinery the mheap ultimately calls is Memory Mapping and OS Interaction. Read Size Classes and Span Management first — this note assumes you know what a span and a size class are.

Mental Model

The hierarchy is a supply chain with three warehouses at increasing distance and cost:

  • mcache — the toolbox on your workbench. One per P, no sharing, no locks. Holds at most one current span per span class. The 99% case: grab a slot from the current span’s bitmap.
  • mcentral — the stockroom down the hall. One per span class, shared by all Ps, guarded by lock-free spanSet structures with per-shard spine locks. Holds lists of partially-full and full spans. Visited only when an mcache’s current span is exhausted.
  • mheap — the central warehouse and loading dock. One global instance. Owns the Page Allocator and talks to the OS. Visited only when a mcentral has no usable span and must mint a fresh one from pages.
flowchart TD
    P1["P 0<br/>mcache"] -.refill.-> MC["mcentral<br/>(one per span class)"]
    P2["P 1<br/>mcache"] -.refill.-> MC
    P3["P 2<br/>mcache"] -.refill.-> MC
    MC -.grow.-> MH["mheap (global)<br/>pageAlloc + arenas"]
    MH -.sysAlloc.-> OS["operating system<br/>mmap / VirtualAlloc"]
    P1 --- A["lock-free<br/>fast path"]
    MC --- B["locked,<br/>amortized per span"]
    MH --- C["locked + OS calls,<br/>amortized per 1 MB+"]

Diagram: the supply chain. The insight is the amortization ladder: each tier serves a unit larger than the tier above it consumes — mcache serves objects, mcentral serves spans (hundreds of objects), mheap serves page-runs (whole spans), the OS serves arenas (≥1 MB). Cost rises at each step, but so does the unit, so the expensive step is taken proportionally less often.

Tier 1 — mcache: the per-P lock-free cache

The mcache struct (runtime/mcache.go, lines 20–66) is small and tuned for cache locality (mcache.go):

type mcache struct {
	_ sys.NotInHeap
 
	nextSample  int64   // bytes until next heap-profile sample
	memProfRate int     // cached MemProfileRate
	scanAlloc   uintptr // bytes of scannable heap allocated
 
	tiny       uintptr  // current tiny block (see Tiny Allocator)
	tinyoffset uintptr  // offset into the tiny block
	tinyAllocs uintptr  // count of tiny allocations
 
	alloc [numSpanClasses]*mspan // current span per span class
 
	reusableNoscan [numSpanClasses]gclinkptr // freegc experiment free lists
 
	stackcache [_NumStackOrders]stackfreelist
	flushGen   atomic.Uint32
}

Key points:

  • _ sys.NotInHeap — an mcache is not on the GC heap. It is allocated by allocmcache from mheap_.cachealloc, a fixalloc free-list allocator (mcache.go, allocmcache). It must not be GC-traced; the one heap pointer it holds (tiny) is special-cased.
  • alloc [numSpanClasses]*mspan — the heart of the fast path. numSpanClasses = NumSizeClasses << 1 = 136: one entry per (size class × {scan, noscan}). Each entry is the current span the P allocates this size class from. On creation every entry points at emptymspan, a shared dummy span with zero free objects, so the first allocation of every class forces a refill.
  • No lock. The comment is explicit: “No locking needed because it is per-thread (per-P).” A goroutine cannot be preempted off its P mid-allocation because mallocgc sets mp.mallocing = 1 (acquirem), pinning the M to its P for the duration.
  • flushGen — records the sweep generation when the cache was last flushed. If it falls behind mheap_.sweepgen, the cached spans are stale and must be flushed before use; this is checked in acquirep when a P is bound to an M.

The fast path

For a small noscan allocation, mallocgc calls mallocgcSmallNoscan (malloc.go). It:

  1. acquirem() and sets mp.mallocing = 1 — pins to the P, blocks preemption.
  2. Gets c := getMCache(mp) and span := c.alloc[spc] for the computed span class.
  3. Calls nextFreeFast(span) — a ctz on the span’s inverted allocCache (see Size Classes and Span Management) returning a free slot, or 0 if the cache window is empty.
  4. If nextFreeFast returns 0, calls c.nextFree(spc), which scans the rest of the bitmap and, if the span is fully exhausted, calls c.refill(spc).
  5. Releases the M.

Steps 1–3 are the lock-free common path: no atomics, no mheap lock, just a bit scan on memory the P owns.

refill — returning a full span, getting a fresh one

When the current span is full, mcache.refill swaps it for a new one (mcache.go, lines 160–239):

func (c *mcache) refill(spc spanClass) {
	s := c.alloc[spc]
	if s.allocCount != s.nelems {
		throw("refill of span with free space remaining")
	}
	if s != &emptymspan {
		// Hand the now-full span back to the mcentral.
		mheap_.central[spc].mcentral.uncacheSpan(s)
		// ... update allocation stats, flush tinyAllocs ...
	}
	// Get a new span with free slots from the mcentral.
	s = mheap_.central[spc].mcentral.cacheSpan()
	if s == nil {
		throw("out of memory")
	}
	s.sweepgen = mheap_.sweepgen + 3 // mark as cached, prevent async sweep
	s.allocCountBeforeCache = s.allocCount
	gcController.update(int64(s.npages*pageSize)-int64(uintptr(s.allocCount)*s.elemsize), int64(c.scanAlloc))
	c.alloc[spc] = s
}

Two important details. First, refill over-counts: it tells the GC pacer (gcController.update) to assume all slots of the new span will be used (an overestimate), and fixes the estimate down later in releaseAll if slots go unused — the comment explains an underestimate would mislead the pacer into thinking the heap is healthier than it is (issue #53738). Second, s.sweepgen = sweepgen + 3 is a special state in the span sweep-generation state machine meaning “cached and swept” — it prevents the background sweeper from touching a span that a P is actively allocating from.

Tier 2 — mcentral: the per-class shared list

There is one mcentral per span class, stored in mheap_.central[numSpanClasses]. Its job is to hold spans that are not currently cached by any P and hand them out / take them back (mcentral.go):

type mcentral struct {
	_         sys.NotInHeap
	spanclass spanClass
	partial [2]spanSet // spans with at least one free object
	full    [2]spanSet // spans with no free objects
}

Why two spanSets each for partial and full? Because of sweeping. Go’s sweeper is lazy — a span is not swept until its slots are next needed. mcentral keeps a swept set and an unswept set, and they swap roles every GC cycle: partialSwept(sweepgen) returns partial[sweepgen/2%2], partialUnswept(sweepgen) returns partial[1−sweepgen/2%2]. sweepgen advances by 2 per cycle, so the two indices alternate (mcentral.go, lines 57–79). A spanSet is itself a lock-free-ish stack with a sharded “spine” guarded by spineLock — so even the mcentral is not a single fat mutex.

cacheSpan — finding a span with free slots

cacheSpan is the slow path’s heart. It tries, in order (mcentral.go, lines 82–199):

  1. Partial swept — c.partialSwept(sg).pop(). A span already swept and known to have free slots: best case, use it directly.
  2. Partial unswept — pop spans from the unswept partial set, sweep each one (s.sweep(true)), and use the first that yields free space. Bounded by spanBudget = 100 to cap latency: “If we sweep spanBudget spans without finding any free space, just allocate a fresh span… By setting this to 100, we limit the space overhead to 1%.”
  3. Full unswept — pop spans from the unswept full set; sweeping one may free objects (the GC marked some dead), turning a “full” span into one with free space. If sweeping yields nothing, push it onto the swept-full list.
  4. Grow — if all of the above fail, c.grow() asks the mheap for a brand-new span.

After obtaining a span, cacheSpan calls s.refillAllocCache and shifts allocCache so the span’s freeindex aligns with bit 0 — priming the per-P fast path. The tryAcquire/sweepLocker dance handles a subtle race: an asynchronous background sweeper may have grabbed a span that is still on the unswept list; cacheSpan must detect that and skip it rather than double-sweep.

uncacheSpan — taking a span back

When a P’s mcache is done with a span (it filled up, or the cache is being flushed), uncacheSpan files it back into the right list (mcentral.go, lines 205–248): if the span still has free slots, onto partialSwept; if full, onto fullSwept; if it was cached before the current sweep began (stale), sweep it immediately.

grow — minting a fresh span

func (c *mcentral) grow() *mspan {
	npages := uintptr(gc.SizeClassToNPages[c.spanclass.sizeclass()])
	s := mheap_.alloc(npages, c.spanclass)
	if s == nil {
		return nil
	}
	s.initHeapBits()
	return s
}

SizeClassToNPages (from internal/runtime/gc/sizeclasses.go) gives the page count for this class — 1 for small classes, up to 10 for some large ones. mheap_.alloc is the call into Tier 3.

Tier 3 — mheap: the global heap

The mheap is the single global owner of all heap memory. The struct (runtime/mheap.go, hundreds of fields) embeds the Page Allocator (pageAlloc), the arena index (arenas), the array of 136 mcentrals, and several fixalloc allocators for off-heap metadata (spanalloc for mspan structs, cachealloc for mcache structs) (mheap.go).

mheap.alloc(npages, spanclass) does, on the slow path:

  1. Take mheap_.lock — a global mutex. This is the most contended lock in the allocator, which is why the upper tiers exist: to reach it as rarely as possible.
  2. Ask the Page Allocator (pageAlloc.alloc) for a run of npages contiguous free pages.
  3. If the page allocator has no run that large, grow the heap: reserve and map a new arena (64 MiB on 64-bit, 4 MiB on 32-bit) from the OS — see Memory Mapping and OS Interaction.
  4. Allocate an mspan struct from spanalloc and initialize it: set startAddr, npages, spanclass, elemsize, nelems, allocate allocBits/gcmarkBits.
  5. Register the span in the arena’s span map so any address can be mapped back to its span.

The mheap also owns large allocations directly: mcache.allocLarge (for objects > 32 KiB) calls mheap_.alloc itself, skips the mcache/mcentral tiers entirely, and pushes the resulting single-object span straight onto the mcentral’s swept-full list so the background sweeper still sees it (mcache.go, allocLarge).

Heap Bits — In-Span vs. Header

A wrinkle that the three tiers must coordinate: the GC needs to know, for every object, which words are pointers. Small objects store this in the span (a packed pointer bitmap alongside the span’s data, heapBitsInSpan(size) is true), while objects above a threshold (MinSizeForMallocHeader, which equals PtrSize * PtrBits — 512 bytes on 64-bit) carry an 8-byte malloc header prepended to the object that points at the type (gc/malloc.go). mallocgc branches on this: mallocgcSmallScanNoHeader vs. mallocgcSmallScanHeader. Noscan objects need neither. This is why roundupsize pads scannable requests by MallocHeaderSize.

Go 1.26 — experiments visible in the tree, and what actually shipped on by default

Two allocator-adjacent experiments are present in the Go 1.26 source but remain off by default. First, mallocgc has a sizeSpecializedMallocEnabled fast path: for small fixed sizes it dispatches through mallocNoScanTable / mallocScanTable, arrays of size-specialized allocation functions, eliminating size-class lookup overhead (malloc.go). Second, the runtimeFreegc experiment (goexperiment.RuntimeFreegc) adds the reusableNoscan [numSpanClasses]gclinkptr free lists to the mcache, letting the runtime reclaim individual noscan objects without a full GC cycle. The reusableNoscan field is present in the struct unconditionally, but the methods that populate and drain it are guarded by runtimeFreegcEnabled (verified in runtime/mcache.go at the go1.26.0 tag).

Both are gated GOEXPERIMENTs and are not in the Go 1.26 default baseline: internal/buildcfg.ParseGOEXPERIMENT at the go1.26.0 tag sets only RegabiWrappers, RegabiArgs, Dwarf5, RandomizedHeapBase64, and GreenTeaGC as defaults — neither SizeSpecializedMalloc nor RuntimeFreegc appears (buildcfg/exp.go @ go1.26.0). The Go 1.26 release notes likewise do not mention either (Go 1.26 release notes). To opt in at build time you would set GOEXPERIMENT=sizespecializedmalloc / GOEXPERIMENT=runtimefreegc.

What did ship on by default in Go 1.26 and touches this hierarchy is the Green Tea garbage collector (GOEXPERIMENT=greenteagc, now default-on, opt-out via nogreenteagc and slated for removal in Go 1.27). It is a collector change, not an allocator change — the three-tier mcache/mcentral/mheap flow described above is unchanged — but it reworks how the marker scans small objects for better locality and CPU scalability (Go expects a 10–40% cut in GC overhead in GC-heavy programs, with extra gains from vector instructions on Ice Lake / Zen 4 and newer) (Go 1.26 release notes). Because the marker walks objects laid out by these size-class spans, the allocator’s per-span layout is precisely what Green Tea exploits — see Go Garbage Collector. The other default-on 64-bit runtime change, heap-base-address randomization (RandomizedHeapBase64), is covered in Memory Mapping and OS Interaction.

Failure Modes and Common Misunderstandings

“The allocator takes a lock on every allocation.” False for the common case. The per-P mcache fast path takes no lock. The mheap global lock is reached only when an mcentral must grow — rare, because each mcentral visit hands out a whole span. Lock contention in allocation usually means either very large allocations (which always hit mheap) or a workload allocating faster than spans can be cached.

“More Ps means more allocator contention.” Mostly the opposite: more Ps means more mcaches, more lock-free fast paths. Contention rises only at the mcentral/mheap tiers, and spanSets are sharded to reduce even that. GOMAXPROCS scaling of allocation is generally good.

“mcache holds all my freed memory.” No. An mcache holds at most one current span per span class — its job is to serve new allocations, not to retain freed objects. Freed memory is reclaimed by the sweeper and returned to mcentral/mheap, not pooled in the mcache. To pool reusable objects explicitly, use sync.Pool.

Stale-span crashes after GC. The flushGen/sweepgen machinery exists to prevent a P from allocating out of a span that the GC has not yet swept for the new cycle. The +1/+3 sweepgen offsets and the prepareForSweep/releaseAll flush in acquirep are the correctness glue; bugs here historically caused use-after-sweep (the s.sweepgen != mheap_.sweepgen+3 throw in refill is a guard).

Alternatives and Comparison

  • TCMalloc — the direct ancestor, also three-tier: per-thread cache → central free list → page heap. Go’s version replaces “per-thread” with “per-P” (so a blocked goroutine does not pin a cache) and threads GC mark/sweep state through every tier (TCMalloc design).
  • glibc ptmalloc — per-thread arenas rather than per-thread caches; arenas are full sub-heaps with their own locks. Coarser-grained and lock-heavier on the fast path than Go’s per-P span cache.
  • jemalloc — per-thread caches (tcache) → per-arena bins → extents. Architecturally the closest non-Go analogue; differs mainly in not needing GC integration.
  • A single global free list — what the hierarchy exists to avoid: every allocation would contend on one lock. The three tiers are an amortization structure layered on top of exactly that idea.

Production Notes

The co-design of allocator and collector is the theme of Rick Hudson’s ISMM 2018 keynote (ismmkeynote): the per-P cache makes allocation cheap and gives the GC a clean place to do allocate-black (marking new objects live during a GC cycle) and lazy sweeping.

Practical guidance:

  • runtime.MemStats / GODEBUG=gctrace=1 report heap-in-use and span counts; a large gap between heap-in-use and heap-released points at spans held by mcentral/mheap that the scavenger has not returned to the OS — see Memory Mapping and OS Interaction.
  • Allocation-rate spikes show up as mheap lock contention in pprof mutex/block profiles. The fix is usually to allocate less (escape analysis, reuse, sync.Pool) rather than to tune the allocator — there are few user-visible knobs.
  • Per-P scaling. Because the fast path is per-P, an allocation-heavy program scales with GOMAXPROCS until the mcentral/mheap tiers saturate; profiling will show when that happens.

See Also

Graph View

Backlinks