Runtime Metrics Package

The runtime/metrics package, added in Go 1.16 (Go 1.16 release notes), is the modern, stable interface for reading numbers the Go runtime knows about itself — heap size, goroutine counts, GC pauses, scheduler latency, mutex contention, CPU time split by class (runtime/metrics docs). It exists to supersede the older runtime.ReadMemStats and runtime/debug.ReadGCStats: those have fixed struct shapes that cannot grow without breaking the Go 1 compatibility promise, some of their fields force an expensive stop-the-world, and they only cover memory. runtime/metrics solves all three by making the metric set discoverable at runtime through string-keyed names, so new metrics ship without API changes and consumers ask only for what they need (proposal #37112). As of Go 1.26 (the published module at the time of writing is go1.26.3) the runtime exposes on the order of 110–120 distinct metrics — roughly 66 named static metrics (the /cgo/, /cpu/, /gc/, /memory/, /sched/, /sync/ families) plus roughly 48 dynamically-spliced /godebug/non-default-behavior/<name>:events counters, one per non-opaque GODEBUG setting (metrics/description.go @ go1.26.3) — and the set grows almost every release.

Mental Model

Think of runtime/metrics as a self-describing key-value store exposed by the runtime, in contrast to ReadMemStats’s frozen struct. With the old API the contract is the Go source: every field is hard-coded, every program that reads it is recompiled when the struct changes, and the struct can only ever describe memory. With runtime/metrics the contract is a naming scheme: each metric is a string like /gc/heap/allocs:bytes, you call metrics.All() to discover what exists in this Go version, and you call metrics.Read() to fill in values. New runtime internals add new names; old programs simply do not ask for them and keep working. The proposal puts it precisely: “Although the set of metrics the runtime exposes will not be stable across Go versions, the API to discover and access those metrics will be” (proposal #37112). That single sentence is the whole design — API stability is decoupled from metric-set stability.

flowchart LR
  subgraph old["runtime.ReadMemStats — frozen struct"]
    MS["MemStats{ Alloc, HeapSys,\nPauseTotalNs, NumGC, ... }"]
  end
  subgraph new["runtime/metrics — discoverable key/value"]
    ALL["metrics.All() []Description\n(names + kinds, this Go version)"]
    READ["metrics.Read([]Sample)\nfills Value for requested names"]
  end
  PROG["your program"] -->|recompile to see\nnew fields| MS
  PROG -->|discover at runtime,\nno recompile| ALL
  ALL --> READ
  READ --> V{{"Value: Uint64 |\nFloat64 | Float64Histogram"}}

Diagram: ReadMemStats hands back a fixed struct whose shape is baked into the binary; runtime/metrics hands back a discoverable list of named metrics plus a typed Read. The insight: making the metric set data-not-code is what lets the runtime add metrics (and change the GC, the scheduler, the allocator internally) without ever breaking a consumer.

Why It Supersedes runtime.ReadMemStats

The proposal that introduced the package, Go issue/design #37112, opens by naming four concrete failures of the legacy APIs — and runtime/metrics was designed feature-by-feature to fix each.

1. Frozen shape. runtime.MemStats is a struct, and the proposal states bluntly that “MemStats is hard to evolve because it must obey the Go 1 compatibility rules.” Adding a field is a visible API change; removing one is forbidden. The struct therefore drags obsolete fields forever — the proposal names EnableGC and DebugGC as fields that are confusing and obsolete but cannot be deleted. It also has hard-coded dimensions: BySize is a fixed array of 61 entries because the allocator once had 61 size classes, and that number can never change in the struct even though the allocator’s size-class table has. The runtime’s internal accounting (the heap model, the scavenger, the pacer, the Green Tea redesign) has changed many times; MemStats could not follow cleanly, and several of its fields became approximations or were quietly redefined.

2. Cost — the stop-the-world tax. Getting a fully consistent MemStats snapshot — every field reflecting the same instant — historically required a stop-the-world (STW). For a monitoring loop scraping every few seconds, paying an STW per scrape is a real, measurable latency tax on every request unlucky enough to overlap a scrape. runtime/metrics’s Read is explicitly designed to be cheap: it does not force an STW, and the docs say it is safe to call concurrently from multiple goroutines provided the Sample slices share no underlying memory.

3. Scope — memory only. MemStats is only memory. The runtime also wants to expose scheduler latency, mutex wait time, CPU time by class, cgo call counts, finalizer/cleanup queue depth, and per-GODEBUG-setting counters. A memory-shaped struct cannot hold any of that. runtime/debug.ReadGCStats patched in a little GC history but is equally frozen and equally narrow.

4. gctrace is not machine-readable. The proposal also indicts the other observability path: GODEBUG=gctrace=1 emits a stderr line per GC cycle whose “format is unspecified and changes frequently,” so “automated metric collection systems ignore gctrace.” runtime/metrics is the machine-readable counterpart — see GODEBUG Diagnostics for the human-readable one.

runtime/metrics keeps two stability guarantees that make it safe to build dashboards on, both stated in the package docs: a metric’s kind (its value type) never changes once published, and values are never NaN or infinity — the package “promises to never produce NaN or infinity floating-point values,” so a consumer never has to defend against them. New Go versions may add metrics and may stop supporting old ones; a removed or never-known metric reads back as KindBad rather than producing an error.

Mechanical Walk-through — The Naming Scheme, All, and Read

Metric names. Every metric is a string of the form /path/to/thing:unit. The package documents the exact grammar with a regular expression: a name is <rooted-path>:<unit> where the path always starts with /, uses lowercase letters and hyphens by convention, and the unit is a machine-parseable series of lowercase English unit names delimited by * or / (metrics/description.go). The path groups related metrics (/memory/classes/..., /gc/..., /sched/..., /cpu/classes/..., /sync/..., /cgo/..., /godebug/...); the unit (bytes, objects, seconds, gc-cycles, goroutines, events, percent, cpu-seconds, and compound units like bytes/second) is part of the identity of the metric. The proposal makes the consequence explicit: because the unit is in the name, “unit changes require name changes” — two metrics differing only in unit are genuinely different metrics, and a backward-incompatible redefinition is forced to take a new name. The unit-in-the-name convention means a dashboard never has to guess what 42 means. The proposal also chose hierarchical strings over integer IDs deliberately: string keys avoid “polluting the API” with a growing list of deprecated named constants, and they support mechanical rewriting and prefix-based grouping/filtering across collection systems.

Discovery — All(). metrics.All() returns []Description, sorted by name. Each Description carries four fields: Name (the full path:unit string), an English Description string, the Kind (KindUint64, KindFloat64, KindFloat64Histogram, or KindBad), and a Cumulative bool saying whether the metric only ever increases (a counter) versus being a gauge that can move up and down. A monitoring agent typically calls All() once at startup to learn the metric set, then loops on Read. Internally the runtime keeps an allDesc slice of the static metrics and an init() function that splices in the per-GODEBUG /godebug/non-default-behavior/<name>:events entries while preserving sort order (metrics/description.go).

Reading — Read(). metrics.Read(m []Sample) takes a slice of Sample (each a Name string plus a Value), and fills each Value in place. You set the Name fields; Read populates the Values. Crucially you pass only the metrics you care about — Read does not return everything, so a loop scraping three metrics costs roughly three metrics’ worth of work, not the whole runtime’s. A name the current runtime does not recognize is populated with a Value whose Kind() is KindBad, which is how you detect a metric that was removed in a newer Go or that you typo’d. The docs are emphatic about two concurrency rules: multiple Read calls are safe only if their argument slices share no underlying memory, and it is a data race to read or manipulate a Value while a Read that touches it is outstanding.

The Value type. Value is a tagged union — the proposal describes it as “emulating sum types.” Internally it embeds a uint64 slot for the scalar kinds and uses an unsafe.Pointer only for the histogram kind, so reading a Uint64 or Float64 metric allocates nothing while a histogram metric can allocate its backing slices. Value is opaque from outside: you ask its Kind(), then call the matching accessor — Uint64(), Float64(), or Float64Histogram(). Calling the wrong accessor panics; the API forces you to branch on Kind rather than guess, trading a little verbosity for the impossibility of silent type confusion.

The histogram kind. A Float64Histogram is {Counts []uint64, Buckets []float64}. Counts[n] is the weight in the half-open range [Buckets[n], Buckets[n+1]), so len(Counts) == len(Buckets) - 1. Buckets[0] is an inclusive lower bound and Buckets[len-1] an exclusive upper bound, and the outermost buckets may be -Inf or +Inf to catch values below or above the instrumented range. The docs add a guarantee that matters for long-running scrapers: bucket boundaries do not change over the lifetime of a program, so a downstream system can cache them. Histograms are how the runtime exposes distributions — scheduler latency, GC pause length, allocation-size spread — without forcing the consumer to compute percentiles from raw samples; the proposal introduced the distinct kind precisely so distributions could be exposed type-safely and extensibly rather than as a raw []byte or an interface{}.

Code Examples

Read every metric the runtime exposes

package main
 
import (
	"fmt"
	"runtime/metrics"
)
 
func main() {
	descs := metrics.All()                       // 1 discover the metric set
	samples := make([]metrics.Sample, len(descs)) // 2 one Sample per metric
	for i := range samples {
		samples[i].Name = descs[i].Name           // 3 set names; values are zero
	}
	metrics.Read(samples)                         // 4 fill in all values
 
	for _, s := range samples {
		switch s.Value.Kind() {                   // 5 branch on the value's kind
		case metrics.KindUint64:
			fmt.Printf("%s = %d\n", s.Name, s.Value.Uint64())
		case metrics.KindFloat64:
			fmt.Printf("%s = %f\n", s.Name, s.Value.Float64())
		case metrics.KindFloat64Histogram:
			h := s.Value.Float64Histogram()       // 6 {Counts, Buckets}
			fmt.Printf("%s = histogram(%d buckets)\n", s.Name, len(h.Buckets))
		case metrics.KindBad:
			fmt.Printf("%s = unsupported in this Go version\n", s.Name)
		}
	}
}

Line 1 discovers what this Go version supports — never hard-code the list. Lines 2–3 build the request: a Sample slice with names filled and values left zero. Line 4 is the single Read that populates every value in place. Line 5 branches on Kind because the accessor must match the kind exactly or it panics. Line 6 destructures a histogram. The KindBad arm matters for forward compatibility — code written against Go 1.30’s metric set still runs on Go 1.26, it just sees KindBad for names that do not exist yet. Because descs comes straight from All(), this loop is correct on every Go version with zero changes.

Scrape a fixed set in a monitoring loop

var samples = []metrics.Sample{
	{Name: "/memory/classes/heap/objects:bytes"}, // live + dead-unswept heap objects
	{Name: "/memory/classes/total:bytes"},        // everything the runtime mapped RW
	{Name: "/gc/heap/goal:bytes"},                // next-GC target (the GOGC/pacer goal)
	{Name: "/gc/heap/live:bytes"},                // live bytes marked by the last GC
	{Name: "/gc/cycles/total:gc-cycles"},         // cumulative GC count (a counter)
	{Name: "/sched/goroutines:goroutines"},       // current live goroutine count
	{Name: "/sched/latencies:seconds"},           // histogram: runnable -> running
}
 
func scrape() {
	metrics.Read(samples)                         // cheap: only these 7
	for i := range samples {
		if samples[i].Value.Kind() == metrics.KindBad {
			// metric vanished in this Go version — skip, do not panic
			continue
		}
	}
	heapObjs := samples[0].Value.Uint64()
	total := samples[1].Value.Uint64()
	goal := samples[2].Value.Uint64()
	live := samples[3].Value.Uint64()
	gcs := samples[4].Value.Uint64()
	gs := samples[5].Value.Uint64()
	lat := samples[6].Value.Float64Histogram()
	_, _, _, _, _, _, _ = heapObjs, total, goal, live, gcs, gs, lat
}

This is the production shape: declare the handful of metrics you export to Prometheus/Datadog once, reuse the same samples slice every scrape so there is no per-scrape allocation, and call Read. Because you asked for seven metrics, you pay for seven — not for the ~100 the runtime could expose. The KindBad guard before any typed accessor is what keeps the scraper from panicking when it runs against a Go version that dropped a metric. /sched/latencies:seconds is the canonical reason to reach for runtime/metrics over ReadMemStats: it is a histogram of how long goroutines sat in the runnable state before getting a P, the single best signal for scheduler starvation, and ReadMemStats has nothing remotely like it.

Turn a histogram into a percentile

// pct computes an approximate p-th percentile (p in [0,1]) from a
// runtime/metrics Float64Histogram.
func pct(h *metrics.Float64Histogram, p float64) float64 {
	total := uint64(0)
	for _, c := range h.Counts {                  // 1 total weight
		total += c
	}
	if total == 0 {
		return 0
	}
	target := uint64(p * float64(total))          // 2 rank we are seeking
	cum := uint64(0)
	for i, c := range h.Counts {                  // 3 walk buckets in order
		cum += c
		if cum >= target {
			return h.Buckets[i+1]                 // 4 upper edge of this bucket
		}
	}
	return h.Buckets[len(h.Buckets)-1]
}

Line 1 sums every bucket’s count to get the total number of observations. Line 2 converts the requested quantile into an absolute rank. Line 3 walks the buckets in order — they are guaranteed sorted — accumulating weight. Line 4 returns Buckets[i+1], the exclusive upper edge of the bucket the target rank lands in; that is the off-by-one to remember (Counts[i] covers [Buckets[i], Buckets[i+1])). This is approximate by nature: a histogram only knows which bucket an observation fell in, not its exact value, so the result is bounded by bucket granularity. For /sched/latencies:seconds this is exactly how a dashboard derives “p99 scheduler latency.”

Notable Metric Families

A representative slice of the catalogue (the full, version-specific list is in the package docs; the exact set grows nearly every release):

• /memory/classes/...:bytes — the runtime’s full memory taxonomy. /memory/classes/heap/{objects,free,released,unused,stacks}:bytes, /memory/classes/metadata/{mspan,mcache}/{inuse,free}:bytes, /memory/classes/os-stacks:bytes, /memory/classes/profiling/buckets:bytes, /memory/classes/other:bytes. /memory/classes/total:bytes is the sum — all memory the runtime mapped read-write into the process (excluding memory mapped via cgo or the syscall package).
• /gc/... — /gc/heap/{allocs,frees}:bytes and :objects (cumulative), /gc/heap/goal:bytes (the pacer’s target), /gc/heap/live:bytes (live bytes from the last mark), /gc/heap/objects:objects, /gc/heap/{allocs,frees}-by-size:bytes (histograms), /gc/cycles/{automatic,forced,total}:gc-cycles, /gc/gogc:percent, /gc/gomemlimit:bytes, the /gc/scan/{heap,stack,globals,total}:bytes family, and /gc/limiter/last-enabled:gc-cycle (which GC cycle the GC CPU limiter last fired — a key out-of-memory diagnostic). Newer additions include /gc/finalizers/{queued,executed}:finalizers and /gc/cleanups/{queued,executed}:cleanups, whose difference approximates the finalizer/cleanup queue depth (see Finalizers and runtime.AddCleanup).
• /sched/... — /sched/goroutines:goroutines (live count), /sched/gomaxprocs:threads, /sched/threads/total:threads, /sched/goroutines-created:goroutines (a counter), the state-split /sched/goroutines/{running,runnable,waiting,not-in-go}:goroutines gauges, /sched/latencies:seconds (runnable→running histogram), and the /sched/pauses/{stopping,total}/{gc,other}:seconds histogram family.
• /cpu/classes/...:cpu-seconds — CPU time split into user, idle, gc/{mark/assist,mark/dedicated,mark/idle,pause,total}, and scavenge/{assist,background,total} sub-classes, with /cpu/classes/total:cpu-seconds the sum. The docs warn each value is an overestimate and should be compared only against other /cpu/classes metrics, never against wall-clock time.
• /sync/mutex/wait/total:seconds — approximate cumulative time goroutines spent blocked on sync.Mutex, sync.RWMutex, or runtime-internal locks. Good for spotting a global contention change; use the mutex profile to attribute it to a call site.
• /cgo/go-to-c-calls:calls — cumulative count of Go→C calls (see cgo Internals).
• /godebug/non-default-behavior/<name>:events — one counter per GODEBUG compatibility setting, incremented whenever the non-default (old) behavior is taken. This is how you audit, in production, whether any code path relies on a GODEBUG opt-out before that opt-out is removed in a future Go release (see GODEBUG Diagnostics and GODEBUG and Runtime Configuration Knobs).

The count was checked against the go1.26.3 git tag: the allDesc slice in metrics/description.go holds 66 named static descriptions, and the package’s init() splices in one /godebug/non-default-behavior/<name>:events counter for every non-opaque entry in internal/godebugs.All (~48 at this tag), preserving sort order — putting the total in the 110–120 range. The exact total remains release-specific and version-dependent (a few GODEBUG knobs are added or retired most releases), so for a precise figure on a given toolchain run len(metrics.All()).

Failure Modes and Common Misunderstandings

Calling the wrong Value accessor. Uint64() on a KindFloat64 value panics, as does Float64Histogram() on a scalar. Always branch on Kind() first. This is deliberate API design — the panic makes silent type confusion impossible.

Assuming a metric always exists. Metrics can be removed, and a removed-or-unknown name reads back KindBad, not an error. Code that does samples[0].Value.Uint64() without first checking Kind will panic the moment it runs on a Go version that dropped (or never had) the metric. A deprecation foreshadows exactly such a trap: as of go1.26.3, /gc/pauses:seconds is still present but its description reads “Deprecated. Prefer the identical /sched/pauses/total/gc:seconds” (metrics/description.go @ go1.26.3) — a deprecated metric still reads a real value today but is the most likely candidate to become KindBad in a future release, so code that hard-codes it should still guard. Robust agents check for KindBad before every typed accessor.

Treating cumulative metrics as gauges. A Cumulative: true metric — counters like /gc/cycles/total:gc-cycles, /gc/heap/allocs:bytes, /cpu/classes/total:cpu-seconds, or any /godebug/...:events — only ever increases. To get a rate you must take the delta between two scrapes and divide by the interval. Plotting the raw value gives an ever-rising line, not a rate. Conversely, plotting a delta of a gauge like /gc/heap/goal:bytes is meaningless. The Cumulative flag in the Description is there precisely so an exporter can pick the right treatment automatically.

The sampling race. Read is not atomic across all metrics — it does not stop the world, by design. Two metrics read in the same Read call can reflect slightly different instants, and a metric documented as “approximate” (the /sched/goroutines/{running,runnable,...} family explicitly says it is “not guaranteed to add up” to /sched/goroutines:goroutines) genuinely will not reconcile to the byte. Do not write a dashboard alert that fires when running + runnable + waiting + not-in-go != goroutines; the runtime does not promise that identity.

Mismatched-but-overlapping numbers vs. MemStats. runtime/metrics and ReadMemStats are computed from the same underlying accounting but with different consistency models and grouping. /memory/classes/heap/objects:bytes will be close to MemStats.HeapAlloc, not byte-identical at the same instant. Pick one source per dashboard and stay with it; never alert on the difference between them.

Histogram bucket arithmetic. len(Counts) == len(Buckets) - 1. The classic bug is an off-by-one when indexing — remember Counts[n] covers [Buckets[n], Buckets[n+1]), and the outer buckets may be ±Inf. A naïve “average = sum(midpoints × counts) / total” breaks on infinite edge buckets; clamp them or treat the extreme buckets specially.

Alternatives and When to Choose Them

• runtime.ReadMemStats — still in the standard library and still works; reach for it only for the handful of fields with no runtime/metrics equivalent, or in code that must build on pre-1.16 Go. For anything new, prefer runtime/metrics.
• runtime/debug.ReadGCStats — legacy GC history; fully superseded by the /gc/... family.
• pprof and Profiling — for where (stacks, hotspots, contention call sites). runtime/metrics gives aggregate counters and distributions, not stacks. Detect a regression with metrics; attribute it with a profile.
• The Go Execution Tracer — for an exact timeline of scheduler and GC events. Use metrics to detect a regression, a trace to explain when and why it happened.
• schedtrace — human-readable stderr lines for ad-hoc debugging. runtime/metrics is the machine-readable, dashboard-friendly equivalent the proposal was explicitly written to provide, because “automated metric collection systems ignore gctrace.”
• expvar — exposes arbitrary app variables as JSON over HTTP; orthogonal, and you can publish runtime/metrics values through expvar or a Prometheus exporter.

Production Notes

The standard production pattern is a small exporter that calls metrics.All() once at startup, picks the metrics it wants on a dashboard, and scrapes them on the monitoring agent’s interval into a reused Sample slice. Most teams never write that loop themselves: the official prometheus/client_golang Go collector reads from runtime/metrics for exactly this reason, exposing the /gc/..., /sched/..., /memory/... and CPU families as go_* Prometheus series. The metric set is intentionally implementation-defined and grows every release — the GC guide leans on runtime/metrics (especially /gc/heap/goal:bytes, /gc/heap/live:bytes, and the pause histograms) to teach GC tuning, so when you tune GOGC or GOMEMLIMIT these are the numbers you watch. The /gc/limiter/last-enabled:gc-cycle metric is a specific operational gem: if it is nonzero, the GC CPU limiter has fired, which usually means the process was thrashing near an out-of-memory condition — a high-signal alert. Because Read does not stop the world, scraping every second is cheap enough for always-on production monitoring, which was the entire point of replacing ReadMemStats. One caution for long-lived tooling: the metric set is not stable across versions, so an exporter must tolerate KindBad gracefully rather than assuming a metric it knew about last year still exists.

See Also

• pprof and Profiling — stack-level profiling; complementary to aggregate metrics
• The Go Execution Tracer — exact timeline; detect with metrics, explain with a trace
• GODEBUG Diagnostics — /godebug/non-default-behavior/* counters mirror GODEBUG opt-outs
• GODEBUG and Runtime Configuration Knobs — the GODEBUG compatibility mechanism whose counters appear here
• GC Tuning and Observability — uses the /gc/... metric family
• GC Pacing and GOGC — /gc/heap/goal:bytes is the pacer’s target
• GMP Scheduler Model — /sched/latencies:seconds measures its starvation
• Stop-the-World Pauses — the STW tax that runtime/metrics was built to avoid
• Finalizers and runtime.AddCleanup — the /gc/finalizers/* and /gc/cleanups/* queue-depth metrics
• Go Internals MOC — parent map of content