vault

Go Modules

13 min read

Go Modules

A module is the unit of versioning, distribution, and dependency in Go: a tree of Go packages rooted at a directory containing a go.mod file, released and versioned together under a single module path. The module system — introduced experimentally in Go 1.11 as “vgo” and the default since Go 1.16 (Go 1.16 release notes) — replaced the old GOPATH model in which every package lived in one global workspace with no version information. Modules give Go reproducible builds: from a go.mod and a go.sum, any machine resolves the identical set of dependency versions. This note is the entry point to the module system; the file formats, the version-selection algorithm, the proxy infrastructure, and workspaces each have their own deeper note.

Mental Model

A module sits between two scales. Below it are packages — the unit of compilation and import. Above it is the module graph — the directed graph of which module versions require which others. The module is the unit that has a version; individual packages do not.

flowchart TD
    subgraph M["Module  example.com/app  (v1.4.0)"]
      P1[package app] --- P2[package app/internal/db]
    end
    M -->|require v1.9.1| D1["golang.org/x/text"]
    M -->|require v0.27.0| D2["golang.org/x/sys"]
    D1 -->|require| D3["golang.org/x/tools"]
    GOMOD[go.mod: module path + go version + require list]
    GOSUM[go.sum: cryptographic hashes of every dependency]
    M -.described by.-> GOMOD
    M -.verified by.-> GOSUM

Diagram: one module, its packages, its place in the dependency graph, and the two files that pin it down. The insight is the separation of concerns — go.mod records the graph (what is required), go.sum records the content (that what was downloaded is what was expected), and the actual set of versions used is computed from the graph by Minimal Version Selection, not stored.

What Problem Modules Solve

Before modules, Go used GOPATH: all source lived under one directory tree, an import path was just a filesystem path, and there was no notion of a version. Two projects needing different versions of the same library could not coexist; “it worked yesterday” depended on whatever happened to be checked out. The community patched over this with tools like dep, glide, and godep, each with its own lock file.

Russ Cox’s “vgo” proposal (research.swtch.com/vgo-intro) folded versioning into the language toolchain itself, built on three principles: import compatibility (a given import path always means the same thing — so a backward-incompatible change requires a new path), semantic versioning (versions are vMAJOR.MINOR.PATCH and the meaning of each component is a promise), and minimal version selection (the build uses the minimum version satisfying all requirements, making builds reproducible without a lock file).

Mechanical Walk-through

Identity: module path and import path

A module is named by its module path, declared in go.mod (e.g. golang.org/x/text). Every package within the module has an import path that is the module path plus the package’s subdirectory (golang.org/x/text/language). The module path is also, by convention, the location from which the module can be fetched — it usually maps to a VCS repository URL.

For major version 2 and above, the import compatibility rule requires the major version to appear as a suffix in the module path: example.com/lib/v2, example.com/lib/v3. This is semantic import versioning — because v2 is allowed to break the API of v1, and an import path must always mean the same thing, v2 must have a different path. A practical consequence: a single build can simultaneously depend on lib and lib/v2 as two distinct modules (using-go-modules). v0 and v1 carry no suffix (v0 because it makes no stability promise; v1 because it is the baseline).

Creating and growing a module

go mod init example.com/app writes a minimal go.mod. As of Go 1.26, go mod init no longer stamps the current toolchain version into the go directive — running a 1.N.X toolchain produces go 1.(N-1).0 (and a pre-release produces go 1.(N-2).0), to “encourage the creation of modules that are compatible with currently supported versions of Go” (Go 1.26 release notes). To pick a specific version afterward, go get go@version.

From then on the toolchain maintains the file automatically. When go build, go test, or go run encounters an import of a package not yet required, the go command finds the providing module, picks a version (newest tagged release), adds a require line, downloads the source, and records hashes in go.sum — all without explicit user action (using-go-modules). See go.mod and go.sum for the file formats.

The build list

For any build, the go command runs MVS over the module graph to compute the build list — the exact version of every module that will be compiled in. go list -m all prints it. The build list is deterministic and not stored in a lock file; it is recomputed every build from go.mod files, and MVS guarantees the same inputs always yield the same list.

Versions: tagged releases and pseudo-versions

A module version is a semantic-versioning string vMAJOR.MINOR.PATCH, optionally with a -prerelease suffix and +build metadata. The v prefix is mandatory; MAJOR bumps signal breaking changes, MINOR backward-compatible additions, PATCH fixes. A version with major number 0, or any pre-release suffix, makes no compatibility promise.

Not every useful commit is tagged, so the toolchain also synthesizes pseudo-versions for untagged revisions. A pseudo-version such as v0.0.0-20240115093200-daa7c04131f5 encodes three things: a base version, a UTC timestamp yyyymmddhhmmss, and a 12-character commit hash. Pseudo-versions sort below the next tagged release and chronologically among themselves, so MVS can order them coherently. go get module@commit-hash or go get module@branch produces a pseudo-version requirement. The +incompatible build tag marks a v2+ repository that has not adopted the /v2 path suffix — a pre-modules library the toolchain tolerates with reduced guarantees.

Module graph pruning and lazy loading (Go 1.17+)

In a module declaring go 1.17 or higher, the module graph is pruned: it includes the immediate requirements of each go 1.17+ dependency but not their transitive requirements (those are reachable on demand). To make this safe, a go 1.17+ go.mod must list every module needed to build its packages — direct and indirect — as explicit require lines (indirect ones marked // indirect). The companion optimization is lazy module loading: the go command loads only the main module’s go.mod first, and pulls in the rest of the graph only if an import cannot be resolved from what it already has (go.dev/ref/mod). Together these make builds of large dependency trees substantially faster.

The go mod Command Family

go mod init example.com/app   # 1 — create go.mod
go mod tidy                   # 2 — sync go.mod/go.sum to actual imports
go mod download               # 3 — pre-fetch modules into the cache
go mod verify                 # 4 — check cached modules against go.sum
go mod graph                  # 5 — print the full module requirement graph
go mod edit -require=x@v1.2.3  # 6 — scripted, low-level go.mod edits
go mod vendor                 # 7 — copy all deps into ./vendor
go mod why example.com/dep    # 8 — explain why a module is needed

Line 1 bootstraps a module. Line 2 — go mod tidy — is the workhorse: it scans the actual source, adds every module that is imported but missing, removes every require that nothing imports, records the checksums needed, and rewrites the file in canonical form. Line 3 warms the module cache without building. Line 4 checks that nothing in the cache was modified after download. Line 5 dumps the dependency graph as module@version module@version edge pairs. Line 6 is for scripts and CI — it edits go.mod without consulting source code. Line 7 materializes a vendor directory. Line 8 traces the import chain that pulls in a given module.

The Module Cache

Downloaded modules live in the module cache, by default $GOPATH/pkg/mod (or $HOME/go/pkg/mod), overridable with GOMODCACHE (go.dev/ref/mod). The cache is content-addressed and shared across all projects on the machine — a module version is downloaded once and reused everywhere. Cache directories are created read-only to prevent accidental edits to dependency source; this is why rm -rf of the cache can fail and why go clean -modcache (or building with -modcacherw) exists. Under the cache, cache/download/ holds the raw zips and metadata as fetched from the proxy, while module@version/ directories hold the unpacked source. Because the cache is shared, a single go mod download populates it for every project, and CI systems commonly cache this directory between runs to avoid re-fetching.

A Worked Module Lifecycle

The end-to-end flow ties the pieces together (using-go-modules). go mod init example.com/hello writes a two-line go.mod. Adding import "rsc.io/quote" to the source and running go test makes the toolchain find the module, pick its newest tagged release, append require rsc.io/quote v1.5.2, fetch the source into the cache, and write hashes for quote and its transitive dependencies into go.sum. go list -m all then shows the full build list — direct dependency rsc.io/quote plus indirects rsc.io/sampler and golang.org/x/text. Later, go get golang.org/x/text upgrades that one module and records it as // indirect (the main module does not import it directly). Removing the import and running go mod tidy drops the now-unused require. Throughout, the developer never hand-edited either file — the toolchain maintained both, and committing them makes the build reproducible for everyone else.

The -mod flag governs how aggressively the toolchain may rewrite go.mod during a build. -mod=readonly (the default since Go 1.16) refuses to modify go.mod and instead reports the change it would need — the safe CI setting. -mod=mod allows automatic updates. -mod=vendor builds strictly from a vendor/ directory with no cache or network access. GOFLAGS=-mod=... sets the default globally.

Failure Modes and Common Misunderstandings

“go.mod is a lock file.” It is not. go.mod records minimum requirements (a graph); the resolved versions are computed by MVS at build time. The closest thing to a lock file is the go.sum integrity list plus the determinism guarantee — there is no separate go.lock.

require lines are minimums, not pins. A require example.com/x v1.2.0 does not force exactly v1.2.0; if some other module in the graph requires v1.3.0, MVS selects v1.3.0. Newcomers from npm/pip expect the listed version to be the used version. See Minimal Version Selection.

Forgetting // indirect discipline. In go 1.17+ modules the graph relies on indirect requirements being listed. Hand-editing go.mod and deleting an // indirect line can break pruning; go mod tidy repairs it.

Major-version path confusion. Importing example.com/lib and expecting v2 features fails — v2 lives at example.com/lib/v2. The compiler error (“no required module provides package”) is unhelpful until you internalize semantic import versioning.

Editing source under the module cache. It is read-only by design; changes there are silently lost or cause permission errors. To modify a dependency, use a replace directive pointing at a local checkout (see go.mod and go.sum) or a workspace.

Alternatives and When to Choose Them

Within Go, the alternative to plain modules is workspaces (go.work) — for developing several interdependent modules together without committing replace directives — and vendoring — for builds that must not touch the network or the proxy. Both layer on top of modules rather than replacing them. The legacy GOPATH mode still exists (GO111MODULE=off) but is effectively dead for new work. Outside Go, the comparison is to npm/Cargo/pip-style resolvers; the deliberate divergence is MVS over newest-compatible selection — see Minimal Version Selection for that argument in full.

Resolution: from import path to module

A detail worth making explicit is how the go command turns an import path it has never seen into a module to require. Given import "golang.org/x/text/language", the toolchain does not know a priori where the module boundary falls — golang.org/x/text is the module and language is a sub-package, but that is not visible from the path alone. It resolves by querying the proxy: it asks for progressively shorter prefixes of the path (golang.org/x/text/language, then golang.org/x/text, …) until one is a module that actually contains the package. For paths that map directly to VCS hosts, a ?go-get=1 HTTP request returns a <meta name="go-import"> tag naming the module root and repository. This is why an import of a package whose module cannot be located fails with “cannot find module providing package” — the prefix search exhausted without a hit. Once resolved, the chosen version (newest tagged release, or a pseudo-version for an untagged default branch) is written as a require line.

Module-Aware Mode and the GOPATH Transition

Modules did not arrive all at once. GO111MODULE controlled the transition: on forces module-aware mode, off forces legacy GOPATH mode, auto chose based on the presence of a go.mod. Since Go 1.16 the default is effectively on everywhere — module-aware mode is no longer conditional on being outside GOPATH (Go 1.16 release notes). New code never needs GO111MODULE; it survives only for building very old codebases. The practical takeaway is that “are we in a module?” is answered by “is there a go.mod at or above the working directory?” — and if not, most go commands now error rather than silently falling back to GOPATH.

Production Notes

In practice the module system disappears into the background, which was Russ Cox’s stated goal — “understandable, predictable, boring.” Real-world friction concentrates in a few places: private modules, where GOPRIVATE must be set so the go command bypasses the public proxy and checksum database for internal repositories (without it, the first fetch of a company-internal module fails trying to reach proxy.golang.org and sum.golang.org); CI reproducibility, where committing both go.mod and go.sum and running go mod verify (or building with -mod=readonly, the default since Go 1.16) catches drift; and major-version upgrades, which because of semantic import versioning are deliberately a code change (new import paths) rather than a silent go.mod bump. A second recurring operational concern is vulnerability response: because MVS never upgrades on its own, a freshly disclosed CVE in a dependency does not get patched until someone runs go get module@fixed — teams operationalize this by running govulncheck in CI to detect affected modules and then applying a reviewed go.mod change. The recommended hygiene is to run go mod tidy before every commit, commit go.mod and go.sum together, build with -mod=readonly in CI, and treat any go.sum change in review as worth a second look — it is the integrity boundary of the whole dependency graph.

See Also

Graph View

Backlinks