vault

The Go Build Cache

15 min read

The Go Build Cache

The Go build cache is a content-addressable store of compilation artifacts that the go command consults before doing any work. Every step of a build — compiling a package, assembling a file, linking an executable, running a test — is modeled as a cacheable action: the toolchain hashes a complete description of the action’s inputs into an action ID, looks that ID up in the cache, and, on a hit, copies the previously produced output instead of recomputing it. The cache is why go build of an unchanged tree finishes in milliseconds, why go test can print (cached) instead of re-running a suite, and why Go 1.10 (the release that introduced it) could finally drop the old pkg/ install directory as the source of incremental-build state. The cache is correct by construction: if any input changes, the action ID changes, and the stale entry is simply never consulted (cmd/go help cache, Go 1.10 release notes).

Mental Model

Think of the build cache as a pure function memoizer wrapped around the entire toolchain. The go command decomposes a build into a directed acyclic graph of actions — “compile package fmt”, “compile package main”, “link the executable”. Each action is, in principle, a deterministic function: given the same source bytes, the same compiler binary, the same flags, the same environment, it produces byte-identical output. Memoization of a pure function needs a key derived from all of its inputs, and that is exactly the action ID — a SHA-256 hash of “a complete description of a repeatable computation (command line, environment variables, input file contents, executable contents)” (cache.go).

There are two distinct hashes in play, and conflating them is the most common misunderstanding. The action ID is the key: hash of the inputs. The output ID is the hash of the result: hash of the bytes the action produced. The cache stores a small index file mapping action ID → output ID, and stores the actual output bytes under the output ID. This two-level scheme buys deduplication: two different actions (say, the same package compiled by two projects with identical settings) produce the same output ID and therefore share one copy of the bytes on disk.

flowchart TD
  subgraph Inputs
    SRC[source file bytes]
    CC[compiler binary hash]
    FLAGS[flags + env vars]
    DEPS[dependency action IDs]
  end
  Inputs --> H1[SHA-256] --> AID[ActionID = cache key]
  AID --> IDX{index lookup<br/>GOCACHE/ab/abcd...-a}
  IDX -- miss --> RUN[run the compiler]
  RUN --> OUT[output bytes]
  OUT --> H2[SHA-256] --> OID[OutputID]
  OUT --> STORE[(GOCACHE/cd/cdef...-d<br/>content-addressed)]
  IDX -- hit --> OID2[stored OutputID] --> STORE
  STORE --> REUSE[copy output, skip compile]

Diagram: an action’s inputs are hashed into an ActionID used as the cache key; a hit yields a stored OutputID pointing at content-addressed output bytes. The insight: the cache key encompasses every input, so there is no way to get a stale hit — a changed input produces a different key, full stop.

Mechanical Walk-through

Where the cache lives. “The default location for cache data is a subdirectory named go-build in the standard user cache directory for the current operating system” — on Linux ~/.cache/go-build, on macOS ~/Library/Caches/go-build, on Windows %LocalAppData%\go-build (help cache). Setting the GOCACHE environment variable overrides this: per the cmd/go documentation, “Setting GOCACHE to an absolute path enables caching. Setting GOCACHE=off disables the build cache” (cmd/go docs). go env GOCACHE prints the current directory, and the path must be absolute. The cache “is safe for concurrent invocations of the go command” — multiple builds can read and write it simultaneously, which is essential because go build ./... itself runs many compile actions in parallel.

Directory layout. Inside GOCACHE, output and index files are sharded into 256 subdirectories named 00 through ff, keyed by the first byte of the hash (cache.go). This sharding keeps any one directory from growing to millions of entries, which would slow filesystem operations. Within a shard, an entry named <hex>-a is an action index file (the small record mapping an action ID to its output ID, output size, and timestamps) and <hex>-d holds the data — the actual output bytes addressed by output ID.

Building the action ID. When the go command needs to compile a package, it constructs the action’s hash incrementally. Conceptually it hashes: the content of every Go source file in the package; the content (hash) of the compile tool binary itself; the exact command line and -gcflags; relevant environment variables (GOOS, GOARCH, GOEXPERIMENT, CGO_ENABLED, and others); the Go release version; and — crucially — the action IDs of every dependency. Because a package’s action ID folds in its dependencies’ action IDs, a change deep in the import graph propagates upward: editing a leaf package changes its action ID, which changes the importer’s action ID, which changes the importer’s importer, all the way to the final link. This recursion is what makes the cache transitively correct.

The lookup. With the action ID computed, the go command opens GOCACHE/<first-byte>/<actionID>-a. If the index file exists and verifies, it yields the output ID; the command then reads GOCACHE/<first-byte>/<outputID>-d and copies (or hard-links, on supporting filesystems) those bytes into the build’s work directory as if the compiler had just produced them. On a miss, the command runs the real tool, hashes the produced output into an output ID, writes the data file, and writes the index file — populating the cache for next time.

Test result caching. The cache is not only for build artifacts. “In package list mode only, go test caches successful package test results to avoid unnecessary repeated running of tests” (cmd/go docs). The match rule is precise: the run must involve “the same test binary and the flags on the command line come entirely from a restricted set of ‘cacheable’ test flags, defined as -benchtime, -coverprofile, -cpu, -failfast, -fullpath, -list, -outputdir, -parallel, -run, -short, -skip, -timeout and -v.” On top of those flags the test action also folds in the relevant environment variables and the contents of any files the test reads through the os package’s instrumented file access. If a test is re-run with an identical action ID and previously passed, go test redisplays the previous output and prints (cached) in place of the elapsed time instead of re-running the binary. Crucially, -count is not in the cacheable set — so “the idiomatic way to disable test caching explicitly is to use -count=1”: passing any non-cacheable flag disables the cache, and -count=1 does so while being a no-op on behavior (1 is the default). Fuzzing also uses the cache: values “that expand code coverage” when passed to a fuzz function are stored in a subdirectory of the build cache, and go clean -fuzzcache removes them.

go run and go tool outputs (Go 1.24+). The set of things the cache stores widened in Go 1.24: “Executables created by go run and the new behavior of go tool are now cached in the Go build cache. This makes repeated executions faster at the expense of making the cache larger” (Go 1.24 release notes). Before this, go run linked a fresh throwaway executable to a temporary directory on every invocation; now the linked binary is itself a cacheable action output, so a second go run of an unchanged program skips the link step. The trade-off the release notes flag is real — caching whole executables is bulkier per entry than caching .a archives, so the cache grows faster on machines that go run a lot.

Trimming. A naive content-addressed cache grows without bound. The go command therefore “periodically deletes cached data that has not been used recently.” The eviction policy, from the source: “we delete entries that have not been used for at least trimLimit (5 days). Statistics gathered from a month of usage by Go developers found that essentially all reuse of cached entries happened within 5 days of the previous reuse” (cache.go). To make “used recently” cheap to track without hammering the disk, the cache updates an entry’s modification time at most once per hour on access, and runs a trim scan at most once per day. Running go clean -cache deletes everything; go clean -testcache removes cached test results only; go clean -fuzzcache removes cached fuzzing values.

Configuration and Diagnostics

The cache exposes several GODEBUG knobs and go env settings. The following commands and settings cover day-to-day interaction with it.

# 1. Where is the cache, and how big is it?
go env GOCACHE
du -sh "$(go env GOCACHE)"
 
# 2. Wipe the build cache entirely (forces a full rebuild next time).
go clean -cache
 
# 3. Wipe only cached *test* results — leaves compiled packages cached.
go clean -testcache
 
# 4. Force a from-scratch build ignoring the cache for THIS invocation
#    (cache is still read for dependencies; -a rebuilds everything).
go build -a ./...
 
# 5. Verification mode: rebuild everything and confirm cached results match.
GODEBUG=gocacheverify=1 go build ./...
 
# 6. Dump every hash input — voluminous, but shows exactly why a key differs.
GODEBUG=gocachehash=1 go build ./mypkg 2>&1 | head -40
 
# 7. Explain test-cache reuse decisions.
GODEBUG=gocachetest=1 go test ./...

Line 1 prints the resolved cache directory; go env -w GOCACHE=/path would persist an override. Line 2’s go clean -cache is the blunt instrument — it should “not be necessary in typical use” because “the build cache correctly accounts for changes to Go source files, compilers, compiler options, and so on” (help cache).

Lines 3 and 4 distinguish two different “force” operations. go clean -testcache (line 3) only invalidates test results — useful when a test depends on something the cache cannot see (a clock, a network service). The -a flag (line 4) forces rebuilding of all packages but does not delete the cache; it still writes fresh entries.

Lines 5–7 are the three diagnostic GODEBUG modes. gocacheverify=1 “causes the go command to bypass the use of any cache entries and instead rebuild everything and check that the results match existing cache entries” — a non-determinism detector. gocachehash=1 “causes the go command to print the inputs for all of the content hashes it uses to construct cache lookup keys”; diffing two runs of this output is the definitive way to find why a build that “should” have been cached was not. gocachetest=1 “prints details of its decisions about whether to reuse a cached test result.”

For the cgo case the cache has a documented blind spot, handled with the -a flag:

# The build cache does NOT see changes to C libraries imported via cgo.
# After editing system C headers/libraries, force a rebuild:
go build -a ./...

“The build cache does not detect changes to C libraries imported with cgo. If you have made changes to the C libraries on your system, you will need to clean the cache explicitly or else use the -a build flag” (help cache). The reason is mechanical: the cache hashes the Go and C source files in the package, but a #include <foo.h> resolved by the system C compiler is not a file the go command enumerates as an input, so a change to /usr/include/foo.h does not perturb the action ID. See cgo Internals for the cgo build pipeline.

External cache backends — GOCACHEPROG

The GOCACHEPROG mechanism is a stable feature as of Go 1.24. The Go 1.24 release notes confirm the graduation directly: “The cmd/go internal binary and test caching mechanism can now be implemented by child processes implementing a JSON protocol between the cmd/go tool and the child process named by the GOCACHEPROG environment variable. This was previously behind a GOEXPERIMENT.” (Go 1.24 release notes). So a feature first available in Go 1.21 only under GOEXPERIMENT=cacheprog became generally available, no experiment flag required, in Go 1.24 — and remains so through the current Go 1.26 release. The go command speaks a small JSON protocol over the child process’s stdin/stdout (get/put requests keyed by action ID) — see go doc cmd/go/internal/cacheprog. This lets organizations back the cache with a shared network store so CI runners and developers reuse each other’s compilation results, the same idea as Bazel’s remote cache.

Failure Modes and Common Misunderstandings

“The cache is stale.” It almost never is. The action ID folds in every input, so a genuine stale hit would require a hash collision in SHA-256. When a developer believes the cache is stale, the real cause is an input the cache cannot see: a cgo C library (above), a code generator run outside go generate’s view, a file read by a test via a path the test framework’s instrumented os layer did not record, or GOFLAGS differing between invocations. GODEBUG=gocachehash=1 resolves every such case by showing the literal hash inputs.

Tests that “won’t re-run.” go test ./... printing (cached) after you “obviously changed something” means the change did not enter the test action’s ID. Editing a file the test reads via a hard-coded absolute path, or depending on wall-clock time, both evade the cache. go test -count=1 forces a re-run not because it changes the action ID but because -count is outside the cacheable-flag set — supplying any non-cacheable flag turns off result caching for that invocation entirely.

Cache size growth in CI. Ephemeral CI containers start with an empty cache every run, so the cache provides zero benefit unless GOCACHE is mounted on a persisted volume or backed by GOCACHEPROG. Conversely, a long-lived CI host accumulates entries; the 5-day trim eventually bounds it, but a host that builds thousands of distinct commits per day can still hold tens of gigabytes between trims. Mounting GOCACHE on a sized volume and letting trimming run is the supported approach; manually rm -rf-ing it mid-build is unsafe only if a concurrent go process is writing.

Non-deterministic builds break caching silently. If a build embeds a timestamp or a random value into output (for example via -ldflags=-X), the output differs run to run but the action ID may not — or, if the timestamp is in the flags, the action ID differs every time and nothing is ever a hit. GODEBUG=gocacheverify=1 catches the first variant by rebuilding and detecting that results no longer match.

The build cache is distinct from, and complementary to, the module cache (GOMODCACHE, default $GOPATH/pkg/mod), which stores downloaded source for module dependencies and is content-verified against go.sum (Go modules reference). The module cache answers “what are the source bytes of dependency X@v1.2.3”; the build cache answers “what is the compiled output of action Y”. See Module Proxy and Checksum Database and Go Modules.

Before Go 1.10 there was no build cache: incremental builds relied on .a archive files installed into $GOPATH/pkg/GOOS_GOARCH/, and staleness was decided by file modification times — fragile, since mtime does not capture flag or compiler changes, and it forced go install as a prerequisite of fast rebuilds. The content-addressed cache made go build fast on its own and removed the need to install packages. The old -i flag (install dependencies) was deprecated and later removed because the cache made it pointless.

Outside the Go toolchain, Bazel with rules_go performs the same memoization at the level of Bazel actions and supports a distributed remote cache out of the box; GOCACHEPROG brings the distributed-cache idea to the native go command without adopting Bazel. ccache is the analogous tool for C/C++ — and the conceptual ancestor of all of these.

Production Notes

The single highest-leverage use of the build cache in practice is persisting GOCACHE across CI runs. GitHub Actions’ actions/setup-go and equivalent CI integrations cache GOCACHE (and GOMODCACHE) keyed by a hash of go.sum; this routinely cuts CI build-and-test wall time by large factors because unchanged packages and unchanged tests are skipped entirely. The key must include the Go version, because the compiler binary’s hash is part of every action ID — a cache populated by Go 1.25 yields zero hits under Go 1.26.

The Go 1.26 release notes flag a layout change relevant to anyone analyzing binaries rather than the cache itself: “various section reorganizations (.go.module, .gopclntab, etc.)” in produced executables (Go 1.26 release notes) — see Go Executable Layout. This does not change cache behavior but is a reminder that the output an action produces evolves between releases, which is precisely why the Go version is hashed into every action ID.

A subtle correctness property worth internalizing: because dependency action IDs are folded in transitively, the cache makes go build reproducible across machines for a fixed toolchain and source tree — two developers on the same Go version building the same commit produce byte-identical objects and therefore identical output IDs. This is the foundation that makes a shared GOCACHEPROG backend safe: an output produced on machine A is provably the output machine B would have produced.

See Also

Graph View

Backlinks