vault

The net Package and the Poller

13 min read

The net Package and the Poller

The net package gives Go programs a blocking, synchronous I/O API — conn.Read(buf) blocks the calling goroutine until bytes arrive — while underneath every socket is non-blocking and the goroutine that “blocks” is actually parked by the scheduler and resumed by the runtime’s network poller. This note is about the layer that performs that illusion: the net.netFD wrapper, the internal/poll.FD it embeds, the pollDesc hand-off to the runtime, and the deadline mechanism. It is a deliberate sibling of Network Poller, which covers the runtime netpoll machinery itself (the epoll/kqueue/IOCP event loop, integration with the scheduler, the sysmon-driven polling). This note does not re-derive that machinery — it explains the standard-library plumbing that sits on top of it and hands work down to it.

Mental Model

Think of network I/O in Go as a four-layer stack. At the top is the public net API — Conn, Listener, TCPConn, Dialer — that user code touches. Below it is netFD, the net package’s own per-connection struct that knows about addresses, address family, socket type, and connection state. Below that is internal/poll.FD, a runtime-shared, package-agnostic file-descriptor wrapper that implements the actual non-blocking-retry loop and serializes access. At the bottom, reached only through a thin pollDesc handle, is the runtime network poller — the OS event mechanism plus the scheduler hooks.

The crucial design point: the layers above the runtime see a blocking API, and the layer that talks to the kernel uses a non-blocking socket. The conversion happens in internal/poll.FD: it issues a non-blocking syscall, and if the kernel says “would block” (EAGAIN), instead of busy-looping or blocking the OS thread, it calls into the runtime to park the goroutine until the poller reports the fd is ready, then retries.

graph TD
    UC["user code: conn.Read(buf)"] --> NC["net.TCPConn / net.conn"]
    NC --> NFD["net.netFD<br/>(family, sotype, laddr, raddr)"]
    NFD --> IPF["internal/poll.FD<br/>(non-blocking retry loop, fdMutex)"]
    IPF -->|EAGAIN: park goroutine| PD["pollDesc handle"]
    PD -.//go:linkname.-> RT["runtime netpoll<br/>epoll / kqueue / IOCP"]
    RT -.fd ready: ready goroutine.-> IPF
    SYSCALL["non-blocking syscall.Read"] -.issued by.- IPF

Diagram: the four layers from public API down to the runtime poller. The insight: net.netFD is a thin semantic wrapper; the real blocking-over-nonblocking conversion lives one layer down in internal/poll.FD, and the only connection to the runtime is the tiny pollDesc handle reached via //go:linkname.

Mechanical Walk-through

netFD — the net package’s connection struct

Every net.Conn returned by Dial or Accept is, concretely, a *net.conn (or *TCPConn, *UDPConn, etc.) wrapping a *netFD. The struct is small (src/net/fd_posix.go):

type netFD struct {
    pfd         poll.FD   // the real I/O machinery
    family      int       // AF_INET, AF_INET6, AF_UNIX
    sotype      int       // SOCK_STREAM, SOCK_DGRAM
    isConnected bool       // handshake done (for TCP)
    net         string     // "tcp", "udp", "unix", ...
    laddr       Addr       // local address
    raddr       Addr       // remote address
}

netFD carries network semantics — what address family, which endpoints, the protocol name used in error messages — and nothing about how I/O actually waits. All of that is delegated to the embedded pfd poll.FD. The netFD.Read method is therefore a thin wrapper:

func (fd *netFD) Read(p []byte) (n int, err error) {
    n, err = fd.pfd.Read(p)             // delegate to internal/poll
    runtime.KeepAlive(fd)               // keep netFD alive across the call
    return n, wrapSyscallError(readSyscallName, err)  // network-flavored error
}

Line by line: fd.pfd.Read(p) does the actual work in the internal/poll layer. runtime.KeepAlive(fd) prevents the garbage collector from finalizing and closing the fd while a syscall on it is in flight — without it, an aggressive optimizer could decide fd is dead after extracting pfd and let a finalizer close the descriptor mid-read. wrapSyscallError rewraps the low-level errno (EAGAIN, ECONNRESET, …) into the net-package error shape — this is why net errors say read tcp 10.0.0.1:443->10.0.0.2:51000: connection reset by peer rather than a bare errno. netFD.Write, readFrom, writeTo follow the identical delegate-keepalive-wrap pattern.

internal/poll.FD — where blocking becomes non-blocking

internal/poll.FD is the package shared between net and os for all pollable descriptors (src/internal/poll/fd_unix.go):

type FD struct {
    fdmu          fdMutex  // serializes Read/Write, coordinates Close
    Sysfd         int      // the actual OS file descriptor (immutable until Close)
    SysFile                // embedded, platform-dependent fd state
    pd            pollDesc // handle into the runtime poller
    csema         uint32   // close semaphore
    isBlocking    uint32   // non-zero if the fd was set to blocking mode
    IsStream      bool      // stream (TCP) vs packet (UDP) semantics
    ZeroReadIsEOF bool     // false for message-based sockets (UDP)
    isFile        bool     // file vs network socket
}

(SysFile is an embedded struct holding platform-dependent descriptor state — it differs between Unix and Windows builds and is not a flat field. The field set above is verbatim from src/internal/poll/fd_unix.go at the Go 1.26 / master tree.)

The heart of the matter is the Read method. This is the body verbatim from src/internal/poll/fd_unix.go at the Go 1.26 tree:

func (fd *FD) Read(p []byte) (int, error) {
    if err := fd.readLock(); err != nil { return 0, err }    // 1
    defer fd.readUnlock()
    if len(p) == 0 { return 0, nil }                          // 2
    if err := fd.pd.prepareRead(fd.isFile); err != nil {      // 3
        return 0, err
    }
    if fd.IsStream && len(p) > maxRW { p = p[:maxRW] }        // 4
    for {                                                     // 5
        n, err := ignoringEINTRIO(syscall.Read, fd.Sysfd, p)  // 6
        if err != nil {
            n = 0
            if err == syscall.EAGAIN && fd.pd.pollable() {    // 7
                if err = fd.pd.waitRead(fd.isFile); err == nil {
                    continue                                  // 8
                }
            }
        }
        err = fd.eofError(n, err)                             // 9
        return n, err                                         // 10
    }
}
  1. readLock() takes the fdmu so two goroutines cannot interleave reads on the same fd, and so a concurrent Close is coordinated rather than racing.
  2. A zero-length read returns immediately — no syscall.
  3. prepareRead() resets the poller state for this fd before the syscall — it tells the runtime “I am about to read; if I have to wait, here is the slot to wake.”
  4. Stream sockets cap a single read at maxRW (1 GiB − 1 on most platforms) to avoid passing the kernel a length it rejects; this is why a Read on a TCP conn can return fewer bytes than len(p) even when more are buffered.
  5. The retry loop. The key realization: because the socket is non-blocking, syscall.Read returns immediately — either with data, or with EAGAIN.
  6. ignoringEINTRIO issues the actual read(2), transparently retrying on EINTR (signal interruption).
  7. If the kernel returned EAGAIN (“no data, would block”) and the fd is genuinely pollable, this is the non-blocking-over-blocking conversion point.
  8. waitRead() is the magic: it parks the goroutine (not the OS thread) until the runtime poller reports the fd is readable, then the loop retries the syscall. The OS thread is freed to run other goroutines while this one waits.
  9. eofError(n, err) collapses the stream-vs-packet EOF distinction in one place: for a stream descriptor with ZeroReadIsEOF, a zero-byte read becomes io.EOF; for a packet socket it does not.
  10. On real data or a real error (ECONNRESET, EOF, …), return.

This is the precise sense in which Go offers “blocking I/O over non-blocking sockets”: the goroutine blocks at step 6; the thread does not; the socket is non-blocking the whole time.

The pollDesc hand-off to the runtime

pollDesc is the bridge to the runtime poller. It is almost nothing — a single word (src/internal/poll/fd_poll_runtime.go):

type pollDesc struct {
    runtimeCtx uintptr   // opaque handle to a runtime *pollDesc
}

The internal/poll package cannot import runtime, so it reaches the runtime’s poller functions via //go:linkname — compiler directives that bind a local declaration to a symbol defined elsewhere:

  • pd.init() calls runtime_pollOpen(uintptr(fd.Sysfd)). The runtime registers the fd with epoll/kqueue/IOCP and returns an opaque handle, stored in runtimeCtx. This is the moment the fd becomes “known to the poller.”
  • pd.prepareRead/prepareWrite call runtime_pollReset, clearing any stale ready/timeout state for read or write mode.
  • pd.waitRead/waitWrite call runtime_pollWait, which is where the goroutine is actually parked (the runtime puts the goroutine onto the pollDesc’s waiter list and calls gopark); it returns when the poller marks the fd ready or a deadline fires.
  • pd.setReadDeadline/setWriteDeadline call runtime_pollSetDeadline.
  • pd.close() calls runtime_pollClose, unregistering the fd.

Everything after runtime_pollWait is handed off — the epoll_wait loop, the timer wheel that fires deadlines, the scheduler logic that makes a readied goroutine runnable — all of that is Network Poller territory. The net/internal/poll layer’s job ends at “park the goroutine and ask the runtime to wake it.”

Deadlines — the Cancellation Mechanism

Go’s network I/O has no per-call timeout argument; instead Conn exposes SetDeadline, SetReadDeadline, and SetWriteDeadline (pkg.go.dev/net). A deadline is an absolute time after which I/O fails instead of blocking. The documented semantics matter:

  • The deadline “applies to all future and pending I/O, not just the immediately following call” — setting a read deadline aborts a currently parked Read, not only the next one.
  • A zero time.Time means no deadline.
  • It can be pushed forward repeatedly; this is the documented way to build an idle timeout — extend the deadline after each successful read.
  • When a deadline is exceeded, Read/Write return an error wrapping os.ErrDeadlineExceeded, and that error reports true from a Timeout() method (it satisfies the net.Error timeout contract).

Mechanically, SetReadDeadline walks down the same stack — netFD.SetReadDeadline → poll.FD.SetReadDeadline → pollDesc.setReadDeadline → runtime_pollSetDeadline. The runtime arms an internal timer; when it fires, the runtime marks the pollDesc as timed-out for the relevant mode and wakes any goroutine parked in runtime_pollWait for that fd. The woken waitRead returns an error (not nil), so the retry loop in step 6 above does not continue — it falls through and returns the timeout error. This is why a deadline can interrupt an already-blocked read: the goroutine is parked in the poller, and the deadline timer is exactly a second wake source for that same park point.

Dialer.Timeout and Dialer.Deadline (pkg.go.dev/net) bound the connection-establishment phase similarly, and DialContext additionally lets a context cancellation abort an in-progress dial — useful because a single Dial may try several IP addresses (Happy Eyeballs, RFC 6555, controlled by FallbackDelay).

Code Example — A Read with a Deadline

conn, err := net.DialTimeout("tcp", "example.com:443", 5*time.Second)
if err != nil { return err }                       // 1
defer conn.Close()
 
conn.SetReadDeadline(time.Now().Add(2 * time.Second)) // 2
buf := make([]byte, 4096)
n, err := conn.Read(buf)                            // 3
if err != nil {
    var nerr net.Error
    if errors.As(err, &nerr) && nerr.Timeout() {    // 4
        log.Println("read timed out")
    }
    return err
}
_ = buf[:n]
  1. DialTimeout bounds connection setup; internally it sets Dialer.Timeout and dials.
  2. SetReadDeadline arms the absolute-time deadline; this propagates to runtime_pollSetDeadline, arming a runtime timer for this fd.
  3. conn.Read delegates netFD.Read → poll.FD.Read; the goroutine parks in runtime_pollWait if no data is ready. It is woken by either the fd becoming readable or the deadline timer.
  4. If the deadline fired first, err wraps os.ErrDeadlineExceeded; errors.As to net.Error and checking Timeout() is the idiomatic way to distinguish a timeout from a hard failure like ECONNRESET.

Failure Modes and Common Misunderstandings

“Go uses one OS thread per connection.” No. Go uses one goroutine per connection in the common server pattern, but those goroutines multiplex onto a small number of threads. A connection blocked in Read costs a parked goroutine (a few kilobytes of stack) and a poller registration — not a thread. This is the entire reason a Go server can hold hundreds of thousands of idle connections.

“SetReadDeadline only affects the next call.” It affects pending I/O too — it will abort a goroutine currently parked in Read. Conversely, a deadline you set once and never refresh stays in effect: a later Read on the same conn will immediately fail with ErrDeadlineExceeded because the deadline is already in the past. Idle-timeout code must re-set the deadline before each read.

“Closing a conn from another goroutine while a Read is blocked is a race.” It is explicitly supported. Close coordinates through the fdmu and wakes parked goroutines; the blocked Read returns an error (use of closed network connection). Multiple goroutines may call Conn methods concurrently per the documented contract.

Blocking syscalls that the poller cannot handle. Not every fd is pollable. Regular files on most systems are always “ready” to epoll and so go through the blocking path; DNS via the cgo resolver runs getaddrinfo on a real OS thread (see System Calls and the Scheduler). The pure-Go resolver, by contrast, does its UDP/TCP DNS over pollable sockets and so costs only a goroutine — which is why GODEBUG=netdns=go scales better under heavy concurrent name resolution.

Forgetting runtime.KeepAlive-class lifetime bugs in custom fd code. If you build your own net.Conn over a raw fd via os.NewFile/net.FileConn, mishandling fd ownership can let a Close (or finalizer) fire while a syscall is in flight, reading or writing a recycled descriptor. The standard library’s KeepAlive and fdMutex exist precisely to prevent this.

Alternatives and When to Choose Them

For the vast majority of code the net package’s blocking-goroutine model is the right answer — it is simpler than callback or async/await networking and the runtime makes it scale. You reach for alternatives only at the extremes. golang.org/x/sys/unix plus a hand-rolled epoll loop (or a library like gnet/evio) trades the one-goroutine-per-connection model for an explicit event loop; this can cut per-connection memory and goroutine-scheduling overhead for workloads with millions of mostly-idle connections, at a large cost in code complexity and loss of the straightforward blocking style. io_uring on modern Linux is another frontier — the Go runtime does not use it for general networking as of Go 1.26: the runtime’s netpoll is still readiness-based (epoll on Linux, kqueue on the BSDs/macOS, IOCP on Windows), the Go 1.26 release notes record no netpoll change (go.dev/doc/go1.26), and runtime/netpoll.go on master contains no io_uring path; the only io_uring-based Go netpollers live in third-party libraries, not the standard runtime. For UDP-heavy or zero-copy needs, (*net.UDPConn).ReadFromUDPAddrPort, syscall.Sendmmsg-style batching, and raw sockets via golang.org/x/net/ipv4/ipv6 exist. The honest default: use net, use deadlines and context, and only profile your way to a custom loop if you have measured the goroutine/poller overhead as a real bottleneck.

Production Notes

Real-world Go servers lean hard on the deadline mechanism: net/http’s Server.ReadTimeout, WriteTimeout, and IdleTimeout are implemented by calling SetReadDeadline/SetWriteDeadline on the underlying net.Conn — they are the same mechanism described here, surfaced as HTTP server config. A frequent production incident is the missing deadline: a server that calls conn.Read with no deadline and no IdleTimeout can accumulate goroutines parked forever on half-open connections (a peer that vanished without sending a FIN), a slow goroutine leak that looks like Goroutine Leaks. The fix is always a deadline or an idle timeout. Go 1.26 added Dialer.DialTCP, DialUDP, DialIP, and DialUnix methods that “permit dialing specific network types with context values” (go.dev/doc/go1.26) — context-aware, type-specific dialing without the string network argument.

See Also

Graph View

Backlinks