vault

Memory Alignment and False Sharing

13 min read

Memory Alignment and False Sharing

Hardware reads and writes memory in fixed-size chunks. A CPU loads a word (a naturally aligned multiple of the pointer size) per instruction and moves data between cache and memory one cache line (typically 64 bytes on x86-64) at a time. Alignment is the rule that a value’s address must be a multiple of its size class so a single word access suffices; Go’s compiler guarantees it and inserts padding into structs to enforce it (Go spec — Size and alignment guarantees). False sharing is the performance pathology that arises one level up: when two logically independent variables that different goroutines mutate happen to land in the same cache line, every write by one core forces the cache-coherence protocol to invalidate the other core’s copy of the line — so the cores ping-pong a line they never actually share data through. The cure is deliberate over-alignment: pad hot, independently mutated fields onto separate cache lines. This note covers both the correctness-level rule (alignment) and the performance-level pathology (false sharing) because the second is solved with the same mechanism — padding — that the first one is built on.

Mental Model

Picture memory as a tape divided twice over. The fine division is into words — 8 bytes on a 64-bit machine — and a value of size n must start at an address that is a multiple of its alignment (≤ n, often n itself). The coarse division is into cache lines — 64-byte blocks on x86-64 — which are the unit the cache-coherence protocol (MESI and relatives) tracks. The hardware never owns “half a line”: a core that writes one byte takes exclusive ownership of the whole 64-byte line, and any other core caching that line must drop it.

flowchart TD
    subgraph L["One 64-byte cache line"]
        F1["field A<br/>(goroutine 1 writes)"]
        F2["field B<br/>(goroutine 2 writes)"]
    end
    C1["Core 1"] -->|"write A"| L
    C2["Core 2"] -->|"write B"| L
    L -.->|"each write invalidates<br/>the other core's copy"| PINGPONG["Cache-line ping-pong:<br/>line bounces between cores,<br/>no real data shared"]

Figure: false sharing. Goroutine 1 only touches field A, goroutine 2 only touches field B — they share no data — but because A and B sit in one cache line, the MESI protocol treats every write to either as a conflict and shuttles the line between cores. The insight: correctness is unaffected, but throughput collapses; the fix is to push A and B into separate lines with padding.

Mechanical Walk-through

Alignment: the language guarantee

Go’s specification pins down sizes and minimum alignments. For numeric types the sizes are fixed: byte/uint8/int8 are 1, uint16/int16 are 2, uint32/int32/float32 are 4, uint64/int64/float64/complex64 are 8, complex128 is 16 (Go spec). The alignment guarantees are stated as minima:

  • For a variable x of any type, unsafe.Alignof(x) is at least 1.
  • For a variable of struct type, the alignment is the largest of the alignments of its fields, but at least 1.
  • For a variable of array type, the alignment equals the element type’s alignment.

In practice the gc compiler aligns each scalar to its own size (an int64 to an 8-byte boundary, an int32 to 4, and so on). To satisfy this for every field and keep the struct itself aligned when placed in an array, the compiler inserts padding — unused bytes — between fields and at the end of the struct. The struct’s total size is rounded up to a multiple of its alignment. This is why field order changes unsafe.Sizeof of a struct — covered in depth in Struct Memory Layout and Alignment, which this note deliberately does not duplicate.

A separate, sharper rule governs 64-bit atomic operations, and it is still in force. The sync/atomic package documentation’s Bugs section states verbatim: “On ARM, 386, and 32-bit MIPS, it is the caller’s responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically via the primitive atomic functions (types Int64 and Uint64 are automatically aligned). The first word in an allocated struct, array, or slice; in a global variable; or in a local variable … can be relied upon to be 64-bit aligned” (pkg.go.dev/sync/atomic). In other words, on 32-bit platforms (386, arm, 32-bit MIPS) a misaligned int64 passed to the raw atomic.AddInt64 panics at runtime, and only the first word of an allocation is guaranteed aligned. Go 1.19 introduced the typed atomics atomic.Int64 / atomic.Uint64, which “are automatically aligned to 64-bit boundaries in structs and allocated data, even on 32-bit systems” (Go 1.19 release notes); they embed an alignment-forcing field so the compiler guarantees correct alignment regardless of position. The current docs explicitly steer callers toward them — every 64-bit primitive (AddInt64, LoadInt64, …) now carries the note “Consider using the more ergonomic and less error-prone Int64.Add instead (particularly if you target 32-bit platforms; see the bugs section)” (pkg.go.dev/sync/atomic). So: new code uses the atomic.Int64 types and never raw atomic.AddInt64 on a bare field.

Cache lines: the hardware reality

Alignment is about correctness and single-access efficiency. Cache lines are about multi-core performance. Go does not detect the real cache-line size at runtime; the runtime hard-codes a per-GOARCH constant. From internal/cpu, the CacheLinePadSize constant is 64 on amd64/386, 128 on arm64 and ppc64/ppc64le, and 256 on s390x (cpu_x86.go, cpu_arm64.go, cpu_s390x.go). The internal/cpu source comments that “there is currently no runtime detection of the real cache line size so we use the constant per GOARCH CacheLinePadSize as an approximation” (cpu.go). Note that the arm64 value of 128 is deliberately conservative: many arm64 chips have 64-byte lines, but Apple silicon and some server parts have 128-byte lines, so Go pads to the larger figure to be safe everywhere.

The runtime exposes a ready-made padding type: internal/cpu.CacheLinePad, defined as struct{ _ [CacheLinePadSize]byte }, “used to pad structs to avoid false sharing” (cpu.go). Because internal/cpu is internal, user code cannot import it; user code rolls its own padding (see Code Examples).

How false sharing actually degrades a program

The cache-coherence protocol (MESI: Modified, Exclusive, Shared, Invalid) tracks ownership per cache line. When core 1 writes a variable, it must hold the containing line in the Modified state, which requires invalidating that line in every other core’s cache. If core 2 is repeatedly writing a different variable in the same line, core 2’s writes likewise invalidate core 1’s copy. The line bounces between the cores’ caches — each access that would have been an L1 hit (~4 cycles) becomes a coherence miss that crosses the interconnect (tens to low hundreds of cycles). No data is logically shared, no lock is contended, the program is correct — it is just running an order of magnitude slower than it should. The classic symptom: a parallel program that scales negatively — adding goroutines makes it slower — with a profile dominated by cache misses on otherwise innocuous struct fields.

Code Examples

Example 1 — false sharing in a per-shard counter, and the fix

package main
 
import "sync/atomic"
 
const shards = 8
 
// BAD: 8 counters packed into 64 bytes -> all share one or two cache lines.
type CountersBad struct {
	n [shards]atomic.Int64 // 8 * 8 = 64 bytes: every Inc invalidates neighbours
}
 
// GOOD: each counter padded to its own 64-byte cache line.
type paddedCounter struct {
	v atomic.Int64
	_ [64 - 8]byte // 56 bytes of padding -> struct is exactly one cache line
}
type CountersGood struct {
	n [shards]paddedCounter
}
 
func (c *CountersBad) Inc(i int)  { c.n[i].Add(1) }
func (c *CountersGood) Inc(i int) { c.n[i].v.Add(1) }

Line-by-line:

  • CountersBad is [8]atomic.Int64 — 64 contiguous bytes, exactly one (sometimes two) cache lines. Eight goroutines each incrementing “their own” counter c.n[i] all hammer the same line; the line ping-pongs and throughput collapses.
  • paddedCounter wraps one atomic.Int64 (8 bytes) and [56]byte of padding so unsafe.Sizeof(paddedCounter{}) == 64. The padding field is unnamed (_) so it cannot be touched and so it does not widen the struct’s API.
  • In CountersGood, n is [8]paddedCounter — 512 bytes, with each counter starting on its own 64-byte boundary. Eight goroutines now write eight independent lines; zero coherence traffic between them.
  • The hard-coded 64 is the x86-64 figure. Portable code should size the pad from unsafe.Sizeof or a build-tag-selected constant; on arm64 you want 128.

Example 2 — padding sized so it survives field changes

type metric struct {
	count atomic.Uint64
	sum   atomic.Uint64
	// pad the *remainder* of the line, computed from the real field sizes:
	_ [64 - 16]byte // 16 = 8 + 8; if you add a field, recompute
}

The robust pattern is [cacheLine - sizeof(realFields)]byte. The sync package’s own poolLocal does exactly this: it embeds the real fields in a poolLocalInternal struct and then pads with pad [128 - unsafe.Sizeof(poolLocalInternal{})%128]byte so the whole poolLocal occupies a clean number of cache lines — the comment in the source reads “Prevents false sharing on widespread platforms with 128 mod (cache line size) = 0” (sync/pool.go, go1.26.0). The mod-by-128 form is self-correcting: change the inner fields and the pad recomputes at compile time. Note the 128, not 64: by padding to a 128-byte boundary the runtime covers both 64-byte (x86-64) and 128-byte (arm64, ppc64) cache lines with one constant.

Example 3 — measuring it

// Run with: go test -bench=. -cpu=8
func BenchmarkFalseSharing(b *testing.B) {
	var c CountersBad
	b.RunParallel(func(pb *testing.PB) {
		i := 0
		for pb.Next() {
			c.Inc(i % shards) // contended line
			i++
		}
	})
}

Pair this with the CountersGood variant and compare. On an 8-core x86-64 machine the padded version commonly runs several times faster, and perf stat -e cache-misses (or Go’s own -test.cpuprofile) shows the unpadded version dominated by last-level-cache misses while the padded version is not. The exact speed-up and cache-miss reduction are hardware-specific — they depend on core count, interconnect, and last-level-cache size — so treat any single reported number as illustrative rather than canonical; community benchmarks typically report the padded variant cutting cache misses by roughly an order of magnitude on contended counters (Kelvin Floresta — False Sharing in Go, a secondary source). The reliable, reproducible signal is directional: padded beats unpadded, and the gap widens as you add cores. Measure on your own hardware before quoting a figure.

Failure Modes / Common Misunderstandings

Padding is invisible until it isn’t. A struct with _ [56]byte is 8× larger than it looks. In a slice of millions of these you have traded cache coherence for cache capacity — the padded array no longer fits in L2. Padding helps write-contended fields accessed by different cores; it hurts read-heavy, single-core traversal of large collections. Pad the few hot, independently mutated fields; never pad bulk data.

False sharing has nothing to do with correctness. The program is perfectly correct with or without padding — atomics still work, locks still protect. The only symptom is performance, and it is easy to misattribute to lock contention. The tell is that there is no shared lock and the fields are logically independent, yet the parallel speedup is poor or negative.

Hard-coding 64 is wrong on arm64. With the migration to Apple silicon and arm64 servers, a [56]byte pad sized for 64-byte lines leaves arm64 (128-byte lines) still false-sharing. Compute the pad from a GOARCH-aware constant.

The _ field can still be reordered away — almost. A trailing _ [N]byte at the end of a struct is real storage and counts toward Sizeof. But beware: if you put padding between two int8 fields, the compiler’s own alignment padding may overlap your intent. Put padding deliberately, measure with unsafe.Sizeof and unsafe.Offsetof.

Atomic misalignment panic on 32-bit. On 386/arm, a bare int64 field that is not 8-byte aligned will panic when passed to atomic.AddInt64. This is an alignment bug, not a false-sharing one — the fix is to use atomic.Int64 (which self-aligns) or to make the int64 the first field. See Atomic Operations in Go.

Alternatives and When to Choose Them

Don’t share the data at all. The cleanest fix for false sharing is not padding but not having adjacent contended fields. Give each goroutine a thread-local-ish value (a per-P shard, a stack-local accumulator) and combine at the end. sync.Pool and the runtime’s per-P caches (see mcache mcentral and mheap) exist precisely to make per-core data the default.

sync/atomic typed values vs raw operations. atomic.Int64 solves the alignment problem for free and reads more clearly than atomic.AddInt64(&x, 1). It does not solve false sharing — you still pad if many atomic.Int64s are co-located and independently hammered.

Sharded counters / expvar-style aggregation. For high-frequency counters, a per-P or per-CPU sharded counter padded to cache lines beats a single contended atomic. This is the standard pattern in metrics libraries.

Just leave it. If a struct’s contended fields are written rarely, or always by the same goroutine, false sharing costs nothing and padding only wastes memory. Pad only after a profiler (perf, pprof with cache-miss events) points at it.

Production Notes

The Go runtime and standard library use cache-line padding deliberately and sparingly. sync.Pool’s poolLocal is padded to a full cache line — the change that introduced this (golang-codereviews thread) shrank poolLocal from spanning three cache lines to one and measurably cut Pool overhead. The scheduler’s per-P structures and several runtime counters are likewise laid out with false sharing in mind. The internal/cpu.CacheLinePad type is the runtime’s own reusable padding primitive.

In application code, false sharing most often shows up in: per-shard maps or counters in high-throughput servers; ring buffers where the producer’s write index and the consumer’s read index sit adjacent; and “stats” structs where many goroutines bump neighbouring fields. The diagnostic workflow is: notice poor or negative parallel scaling → confirm with perf stat -e cache-misses,cache-references or go test -bench with -cpu swept → identify co-located contended fields with unsafe.Offsetof → pad to a cache line → re-measure. Because the Go memory model (see Go Memory Model) says nothing about cache lines, this is purely a performance exercise — but on hot paths it is one of the highest-leverage micro-optimizations available.

See Also

Graph View

Backlinks