vault

String Internals

9 min read

String Internals

A Go string is, at the machine level, a two-word header: a pointer to a backing array of bytes and an int length (runtime stringStruct; Go spec, String types). The spec defines a string value as “a (possibly empty) sequence of bytes” that is immutable — “once created, it is impossible to change the contents of a string.” That immutability is the single fact from which every other property follows: strings can be shared without copying, sliced in O(1), used as map keys, and compared by value, all because no holder of a string can ever mutate the bytes another holder sees. A string is not a sequence of characters and carries no encoding guarantee — it is raw bytes, conventionally but not necessarily UTF-8 (Go blog, Strings, bytes, runes and characters in Go).

This note is about the runtime representation and mechanics of strings. The Unicode/text-iteration angle — what a rune is, how for range decodes UTF-8 — is the sibling note Rune and Byte Semantics; this note cross-links to it rather than re-explaining it.

Mental Model

A string variable is a small struct on the stack (or in another struct); the bytes it describes live elsewhere — in the read-only data segment for a string literal, on the heap, or inside another allocation.

flowchart LR
  subgraph SV["string value (2 words, 16 bytes on 64-bit)"]
    P["str: unsafe.Pointer →"]
    L["len: int = 5"]
  end
  P --> B["backing bytes: 'h' 'e' 'l' 'l' 'o'\n(read-only if a literal)"]
  S2["string value B\n(a substring)"] -.->|"shares same backing array"| B

Diagram: a string header and its backing array. The insight: copying a string copies only the 16-byte header — the bytes are shared, never duplicated. A substring s[1:4] produces a new header pointing into the same array at a different offset and length, which is why slicing a string is O(1) and allocation-free.

Contrast with a slice, which is a three-word header {ptr, len, cap}. A string has no capacity: because it is immutable there is nothing to append to, so the third word is unnecessary. This is the structural reason a string and a []byte are different types despite both describing byte sequences.

Mechanical Walk-through

The header

The runtime defines (runtime/string.go):

type stringStruct struct {
    str unsafe.Pointer // pointer to the first byte of the backing array
    len int            // number of bytes
}

The reflect.StringHeader type historically exposed the same shape (now superseded by unsafe.String/unsafe.StringData). On a 64-bit platform a string value is 16 bytes — two machine words; see Word Size and Architecture Portability. len(s) is a single load of the len word: O(1), and a compile-time constant when s is a constant string (Go spec).

Indexing yields bytes, not characters

s[i] for a string s is the i-th byte — a byte (uint8), valid for i in 0 .. len(s)-1. It is not the i-th character. For ASCII text the distinction vanishes; for UTF-8 text with multi-byte code points, s[i] can land in the middle of a code point. The spec also forbids &s[i] — “It is illegal to take the address of such an element” — precisely because giving out a pointer into the bytes would create a channel to mutate them, breaking immutability. To iterate characters you for range the string (decoding runes) — see Rune and Byte Semantics.

Immutability and where the bytes live

A string literal’s bytes are placed by the linker into a read-only data section. Attempting to write through a pointer obtained via unsafe to a literal’s bytes typically faults (segfault), because the page is genuinely read-only. Strings built at runtime (e.g. from []byte) get heap-allocated backing arrays; they are still type-system immutable — the language gives you no operation to change them — but the memory itself is ordinary writable heap. The guarantee is enforced by the type system, not always by hardware.

Immutability is what makes the cheap operations sound:

  • Assignment / passing copies 16 bytes; the bytes are shared.
  • Substring s[lo:hi] builds a new header {str + lo, hi - lo} pointing into the same backing array — O(1), no allocation. (This means a tiny substring can pin a huge backing array alive; see Failure Modes.)
  • Strings as map keys work because value equality is well-defined and the bytes cannot change underneath the map.
  • Comparison (==, <) is lexicographic byte-wise; the compiler can fast-path equal pointers + equal lengths.

Conversions: string ↔ []byte ↔ []rune

These cross the immutability boundary, so they copy. The spec: “Converting an array of bytes to a string or a string to an array of bytes allocates a new array and copies the bytes. The conversion never fails at run time” (Go spec, Conversions).

  • []byte(s) — allocates a len(s)-byte array, copies, returns a mutable slice. Runtime: stringtoslicebyte.
  • string(b) for b []byte — allocates, copies the bytes, returns an immutable string. Runtime: slicebytetostring. The copy is mandatory: without it, mutating b afterward would mutate the string.
  • []rune(s) — decodes UTF-8, allocating an int32 per code point (so 4 bytes per rune); see Rune and Byte Semantics.
  • string(r) for r []rune — encodes each rune to UTF-8 and concatenates.

The compiler elides the copy in proven-safe cases. The classic one: for i, c := range []byte(s) — the temporary []byte never escapes and is never mutated, so the compiler reuses the string’s bytes directly. Likewise m[string(b)] (a map lookup keyed by a []byte-to-string conversion) is special-cased to avoid the allocation, using slicebytetostringtmp. Small results may also use a static buffer (tmpBuf) or, for single-byte strings, the runtime’s staticuint64s table — slicebytetostring returns a pointer into that table when n == 1, avoiding any allocation for one-byte strings.

unsafe.String and unsafe.StringData

Since Go 1.20, unsafe.String(ptr *byte, len int) builds a string header pointing at existing bytes with no copy, and unsafe.StringData(s) returns the backing pointer. These let you do zero-copy []byte ↔ string — but you are now responsible for the immutability contract: if you unsafe.String over a []byte and then mutate the slice, you have mutated a “string”, and any code that assumed strings are stable (map keys, comparisons, caches) can corrupt silently. They are an escape hatch for hot paths, not a default. See unsafe.Slice and unsafe.String.

Code: Observing the Header

package main
 
import (
    "fmt"
    "unsafe"
)
 
func main() {
    s := "héllo"                              // 1: 'é' is 2 UTF-8 bytes
    fmt.Println(len(s))                       // 2: 6 — BYTES, not 5 characters
    fmt.Printf("%x\n", s[1])                  // 3: c3 — first byte of 'é', not 'é'
    sub := s[2:4]                             // 4: O(1) substring, shares backing array
    fmt.Println(len(sub))                     // 5: 2
 
    b := []byte(s)                            // 6: ALLOCATES + copies 6 bytes
    b[0] = 'H'                                // 7: legal — b is mutable
    fmt.Println(s, string(b))                 // 8: "héllo Héllo" — s unchanged
 
    p := unsafe.StringData(s)                 // 9: *byte to the backing array
    fmt.Printf("%c\n", *p)                    // 10: 'h'
}

Line 2 is the most common surprise: len counts bytes. Line 3 confirms indexing returns a single byte that may be a UTF-8 continuation byte. Line 4’s sub shares s’s array — no copy. Line 6 is the mandatory-copy conversion; line 7 mutating b is fine and line 8 proves s is untouched (separate backing arrays). Lines 9–10 reach the raw pointer via unsafe.

Failure Modes and Common Misunderstandings

len(s) is not character count. It is byte count. For the number of code points use utf8.RuneCountInString(s) (or len([]rune(s)), which also allocates). See Rune and Byte Semantics.

Substring pins the whole backing array. huge := readMegabytes(); tiny := huge[0:10] — tiny’s header still points into huge’s multi-megabyte array, so the GC cannot reclaim it while tiny is reachable. Fix: force a copy with tiny := string([]byte(huge[0:10])) (or strings.Clone, added in Go 1.18, which does exactly this).

string(intValue) is a rune conversion, not a number-to-text conversion. string(65) yields "A" (the rune U+0041), not "65". go vet flags this. Use strconv.Itoa.

Naive concatenation in a loop is O(n²). s += piece allocates a fresh backing array and copies everything each iteration because strings are immutable. Use strings.Builder, which keeps a growable []byte internally and only converts to a string once at the end (with a zero-copy unsafe.String finalization).

unsafe.String over a mutated slice is undefined behavior in spirit. The compiler, the GC, the map implementation, and string comparison all assume string bytes are stable. Violating that via unsafe produces bugs that are nondeterministic and brutal to debug.

Alternatives and When to Choose Them

Use string for any immutable text you read, compare, or key a map on — it is the cheapest to pass and share. Use []byte when you need to mutate bytes in place or do I/O (io.Reader/io.Writer traffic in []byte). Use strings.Builder to assemble a string from many pieces. Use []rune only when you genuinely need random indexed access by code point (e.g. reversing a string correctly) — and remember it costs 4 bytes per code point and a full decode pass. For zero-copy interop on a proven hot path, unsafe.String/unsafe.StringData, with the immutability obligation understood.

Production Notes

The two-word header makes passing strings nearly free, so Go APIs pass string by value liberally — there is no idiom of *string parameters. The dominant real-world pitfalls are the substring-pins-array memory leak (notably when slicing a few fields out of a large parsed buffer and retaining them) and the O(n²) concatenation loop; both show up readily in pprof heap and CPU profiles (see pprof and Profiling). The Go blog’s Strings, bytes, runes and characters in Go is the canonical primer and stresses the load-bearing point: a string is bytes, and “there is no guarantee” those bytes are valid UTF-8 (Go blog).

See Also

Graph View

Backlinks