vault

Go Compilation Model

12 min read

Go Compilation Model

Go’s compilation model is package-at-a-time, separately compiled, statically linked. The unit of compilation is the package, not the file: the gc compiler ingests all the .go files of one package together and emits a single object file. Crucially, each package’s object file carries enough export data about its public interface that, when another package imports it, the compiler “opens exactly one file… the object file identified by the string in the import clause” (Pike 2012) — it never re-reads the imported package’s source, nor the source of its dependencies. The linker then combines all per-package object files plus the runtime into one self-contained native executable. This model is the structural reason Go builds are fast, and understanding it explains why Go has no header files, why import cycles are forbidden, and why a Go binary needs no external runtime.

Mental Model

The C model and the Go model differ fundamentally in what crosses a compilation boundary. In C, a #include textually pastes a header — and that header pastes its headers — so compiling one file re-processes a transitively expanding pile of declarations (Pike measured a 2007 Google C++ binary’s 4.2 MB of source expanding to 8 GB after includes). In Go, compiling package B produces an object file b.o that already contains a digested summary of everything an importer needs to know about B — and transitively about B’s dependencies, to the extent they affect B’s public interface. Importing B is therefore a single O(1) file read, not a transitive source re-parse.

graph TD
    subgraph "C model: transitive textual include"
        CA[main.c] -->|#include| CH1[a.h]
        CH1 -->|#include| CH2[b.h]
        CH2 -->|#include| CH3[c.h]
        CH3 -.->|re-parsed<br/>every time| CX[exponential blowup]
    end
    subgraph "Go model: digested export data"
        GA[package A source] -->|imports| GOB[b.o object file]
        GOB -->|contains digest of| GOC[c export data]
        GA -->|compiler opens<br/>ONE file| GOB
        GOB -.->|O of 1 read| GX[builds scale linearly]
    end

Diagram: the same dependency chain, two models. The insight: Go’s speed is not a faster compiler doing the same work — it is a compiler doing less work, because the export data in an object file makes each import a single bounded read.

Mechanical Walk-through — Source to Executable

Stage 1 — the package, one compiler invocation

The go command groups the .go files of a directory into one package and invokes go tool compile once on the whole set (go command article). Inside, the gc compiler runs its pipeline on the package as a unit (cmd/compile README):

  1. Parsing — source is tokenized (lexical analysis) and parsed (syntax analysis) into a syntax tree by cmd/compile/internal/syntax.
  2. Type checking — types2 (a port of go/types) validates the program against the type rules of the Go specification.
  3. IR construction (“noding”) — the typed syntax becomes the compiler’s own intermediate representation.
  4. Middle end — IR-level optimizations: inlining, Devirtualization, escape analysis.
  5. Walk — high-level constructs are desugared into primitives (channel and map operations become runtime calls; some switches become jump tables).
  6. SSA generation and optimization — IR is converted to Static Single Assignment Form, where machine-independent optimization passes run.
  7. Machine-code generation — SSA is lowered to architecture-specific instructions, register-allocated, and emitted.

The detailed pass-by-pass story is in Go Compiler Architecture; what matters for the model is the output: one object file per package.

Stage 2 — the object file and its export data

A Go object file (Go Object Files and Archives) contains two things: the compiled machine code for the package’s functions, and export data — a serialized description of the package’s public interface. The compiler overview is precise about what that description holds: “The export data file holds all the information computed during compilation of package P that may be needed when compiling a package Q that directly imports P. It includes type information for all exported declarations, IR for bodies of functions that are candidates for inlining, IR for bodies of generic functions that may be instantiated in another package, and a summary of the findings of escape analysis on function parameters.” Export data is what makes separate compilation work. When package A imports package B, the compiler reads B’s export data to type-check A’s use of B — it learns B’s public API without ever parsing B’s source.

Two of those four ingredients are subtle and load-bearing. First, the inclusion of “IR for bodies of functions that are candidates for inlining” is exactly what enables cross-package inlining: the optimizer compiling A can splice a small function from B inline at its call site because B’s object file shipped the function’s intermediate representation, not just its signature. Second, generic function bodies travel too — because a generic function declared in B may need to be instantiated (stenciled for a concrete type) in A, its body must be available across the boundary. Both of these only work because modern Go (since the move to Unified IR) serializes the type-checked program into a single compact byte stream that doubles as both the noder’s input and the export data: per the README, “Unified IR is also involved in import/export of packages and inlining.” The mechanics of that serialization belong to Go Compiler Architecture and Intermediate Representation in the Go Compiler; what matters for the model is that the object-file boundary carries enough digested IR for the importer to type-check, inline, and instantiate without ever touching the imported package’s source.

This is also why import cycles are a compile error: separate compilation requires a topological order — B must be fully compiled (its object file and export data produced) before A, which imports it, can be compiled. A cycle has no topological order, so the model forbids it. The acyclic import graph is not a style rule; it is a structural prerequisite of the compilation model (Go FAQ).

Stage 3 — the action graph and the build cache

The go command computes a directed acyclic action graph from the import graph and compiles packages in dependency order, in parallel up to -p workers. Before running a compile action it computes a content hash over the action’s inputs — the package’s source files, the compiler flags, the Go toolchain version, and the output hashes of dependency actions — and checks the build cache. A hit means the object file is reused and the compiler is never invoked. This is why a one-line change recompiles only that package and its dependents, not the world.

Stage 4 — linking

Once every needed package has an object file, go tool link (The Go Linker) performs the final assembly: it reads all the object files, resolves cross-package symbol references, lays out the code and data sections, links in the runtime package (the scheduler, garbage collector, allocator — present in every Go binary), writes the runtime bootstrap that sets up the stack and starts main, and emits the platform’s executable format (ELF on Linux, Mach-O on macOS, PE on Windows). The default result is a statically linked binary: for pure-Go programs there is no dynamic dependency, not even on libc. This is the model’s payoff at deployment time — a single file you can copy into a FROM scratch container.

Worked Example — Watching the Model Run

Take a two-package program. Package greet (in greet/greet.go):

package greet
 
func Message(name string) string {  // exported: appears in export data
    return "Hello, " + name
}

Package main (in main.go):

package main
 
import (
    "fmt"
    "example/greet"   // compiler will read greet.o's export data, not greet.go
)
 
func main() {
    fmt.Println(greet.Message("world"))
}

Build it with the build plan exposed:

go build -x -work .

-work prints (and keeps) the temporary work directory; -x prints every command. You will see, in order: a go tool compile invocation for greet producing greet.o (also fmt and its dependencies, and the runtime), then a go tool compile for main — and the main invocation is given -importcfg pointing at greet.o, not greet.go. Finally a single go tool link invocation produces the executable. The ordering is forced: greet compiles before main because main imports it.

Now change one line in greet.go and rebuild. Only greet and main recompile — fmt, runtime, and every other dependency are served from the build cache, because their input hashes are unchanged. Change a line in main.go only, and greet is not recompiled at all: nothing greet depends on changed. This selective rebuild is the action graph plus the cache in action.

Inspect what is statically baked in:

go build -o app . && ldd app

On Linux, ldd app for this pure-Go program reports “not a dynamic executable” — the runtime and all packages are linked in. Contrast a cgo program, which links libc dynamically by default (Static and Dynamic Linking in Go).

Common Misunderstandings

“Go recompiles everything on every build.” No — the action graph plus the content-addressed cache mean only packages whose inputs changed (and their dependents) are recompiled. A clean checkout is slow because it is cold; warm incremental builds are fast.

“Go has header files; the import line is like #include.” It is the opposite. #include pastes source text. import names a compiled package whose object file already holds digested export data. No text is pasted, and the import is a bounded read regardless of how large the imported package is.

“go build interprets or JIT-compiles.” Go is ahead-of-time compiled to native machine code by cmd/compile and cmd/link. go run compiles to a temporary binary and executes it — the speed is the compiler’s, not an interpreter’s.

“The compilation model means Go cannot do whole-program optimization.” Mostly true and deliberate: gc optimizes package-at-a-time, with cross-package inlining of small functions enabled by inlinable bodies in export data, but it does not do full link-time whole-program optimization. Profile-Guided Optimization (stable since Go 1.21) is the closest thing — it feeds a runtime profile back into per-package compilation to guide inlining and devirtualization. TinyGo, by contrast, does whole-program compilation in a single step (see gc Compiler vs gccgo vs TinyGo).

“Object files are portable.” They are not. A .o is specific to the GOOS/GOARCH, the exact toolchain version, and the build flags — which is precisely why those values are folded into the build-cache hash. Cross-compiling rebuilds everything for the target (Cross Compilation in Go).

“Export data is just type signatures.” It is more than that. As the compiler overview spells out, export data carries type information for exported declarations plus the IR of inlinable function bodies, the IR of generic function bodies for cross-package instantiation, and escape-analysis summaries for parameters. That is why a one-line change to an inlinable exported function can force recompilation of importers even when its signature is unchanged: the importer baked the old body in, and its input hash now differs.

Alternatives and Contrasts

The package-at-a-time, statically linked model is one point in a design space:

  • C/C++ separate compilation with headers. Same separate-compilation goal, but the textual #include mechanism re-processes declarations transitively, causing the build-time blowup Go was explicitly designed to escape (Pike 2012).
  • Whole-program compilation (TinyGo, some Rust release builds). The whole program is compiled together, enabling aggressive cross-function optimization and dead-code elimination — at the cost of slow, non-incremental builds. TinyGo uses this to produce tiny microcontroller binaries; see gc Compiler vs gccgo vs TinyGo.
  • JIT / bytecode (JVM, .NET, CPython). Compilation deferred to or done at run time, trading startup cost and a required runtime for portability and run-time optimization. Go rejects this for its target of fast-starting, self-contained server binaries.
  • gccgo. A different implementation of the same language that plugs the Go front end into the GCC back end and links against glibc; its build model and linking defaults differ from gc (it does not default to fully static linking). See gc Compiler vs gccgo vs TinyGo.

Choose Go’s model when you want fast incremental builds and trivially deployable static binaries — the common case. Whole-program models win only when binary size or peak optimization outranks build speed.

Production Notes

The model has direct operational consequences. Static linking is why Go is the default language of containers — a FROM scratch image with one binary is the canonical Go Dockerfile. CI build speed is dominated by build-cache reuse: persisting $GOCACHE between runs turns cold builds (which recompile the standard library) into warm ones. Reproducible builds use -trimpath to remove local paths from object files so the cache hash and the binary are environment-independent. The C-to-Go conversion of the toolchain in Go 1.5 (Go 1.5 release notes) removed the last C from the compilation model, making the toolchain self-hosting — building Go 1.N now needs an earlier Go compiler as a bootstrap rather than a C toolchain (install from source). Go 1.5 also extended the linker with build modes for producing C-archive and C-shared libraries, the one place the otherwise-static model deliberately produces dynamically linkable artifacts.

See Also

Graph View

Backlinks