vault

Network Poller

13 min read

Network Poller

The network poller (the runtime’s netpoll) is how Go turns blocking-style network code into event-loop-grade scalability without the programmer ever seeing an event loop. When a goroutine reads from a socket that has no data, it does not block an operating-system thread inside the kernel. Instead the runtime registers the file descriptor with the platform’s I/O readiness mechanism — epoll on Linux, kqueue on the BSDs and macOS, IOCP on Windows, event ports on illumos — and parks the goroutine, freeing its M to run other goroutines. A small amount of runtime code later asks the OS “which descriptors are ready?” and wakes exactly the goroutines whose I/O can now proceed. The result: one program can hold a million idle network connections on a handful of threads (src/runtime/netpoll.go).

Mental Model

There are two ways to wait for I/O. The straightforward one — call a blocking syscall and let the OS thread sleep in the kernel — is what file I/O does (see System Calls and the Scheduler); it works but consumes a thread per blocked operation. The scalable one is readiness notification: ask the OS to tell you when a descriptor becomes readable or writable, and do not occupy a thread while waiting. The network poller is Go’s implementation of the second approach, hidden entirely behind ordinary Conn.Read/Conn.Write calls.

flowchart TD
    G["Goroutine: conn.Read(buf)"] --> NB["internal/poll: try non-blocking read()"]
    NB -->|"data available"| DONE["Return bytes — no parking"]
    NB -->|"EAGAIN: would block"| BLOCK["poll_runtime_pollWait"]
    BLOCK --> NPB["netpollblock: set pollDesc.rg = pdWait"]
    NPB --> PARK["gopark(...): G -> _Gwaiting<br/>M is freed to run other goroutines"]
    POLL["Scheduler / sysmon calls netpoll(delta)"] --> EP["epoll_wait / kevent / GetQueuedCompletionStatus"]
    EP -->|"fd is ready"| NR["netpollready: netpollunblock<br/>pollDesc.rg -> pdReady, return the G"]
    NR --> RDY["G -> _Grunnable, pushed to a run queue"]
    RDY --> RESUME["Some M schedules the G; the read retries and succeeds"]

Figure: the life of a blocked network read. The key insight is the decoupling: the goroutine that wants the I/O (gopark) and the code that learns the I/O is ready (netpoll) are completely separate. The M that ran the goroutine is not blocked — it goes off and runs other work. The poller is consulted opportunistically by the scheduler, so readiness discovery costs nothing until there is genuinely nothing else to do.

Why the Poller Exists

If Go used a blocking syscall for every socket read, a server holding 100,000 mostly-idle connections would need ~100,000 OS threads — one per connection blocked in read(). Threads are expensive: each has a stack, a kernel descriptor, scheduler overhead. The classic answer in C is to write an event loop — epoll_wait in a while loop, with all logic expressed as callbacks (“callback hell”). Go’s design goal was to keep the programming model synchronous — n, err := conn.Read(buf) reads like blocking code — while getting the scalability of an event loop. The network poller is the runtime machinery that makes those two things compatible: the source code blocks, the goroutine parks, but no thread is consumed. This is the single most important reason Go became a popular language for network servers.

The Platform Abstraction

netpoll.go is the platform-independent core. Each operating system provides a small set of functions; the netpoll.go comment lists the contract verbatim (netpoll.go):

  • netpollinit() — “Initialize the poller. Only called once.” On Linux this creates the epoll instance with epoll_create1(EPOLL_CLOEXEC) and additionally creates an eventfd, registering it with the epoll set (EPOLLIN) so that netpollBreak has a descriptor to poke (netpoll_epoll.go).
  • netpollopen(fd, pd) — “Arm edge-triggered notifications for fd.” On Linux this is an EPOLL_CTL_ADD registering fd with EPOLLIN | EPOLLOUT | EPOLLRDHUP | EPOLLET — note EPOLLET, edge-triggered mode.
  • netpoll(delta) — “Poll the network. If delta < 0, block indefinitely. If delta == 0, poll without blocking. If delta > 0, block for up to delta nanoseconds. Return a list of goroutines built by calling netpollready.”
  • netpollBreak() — “Wake up the network poller, assumed to be blocked in netpoll.”
  • netpollclose(fd) and netpollIsPollDescriptor(fd) — teardown and identification.

Linux uses epoll, the BSDs and macOS use kqueue, illumos/Solaris use event ports, Windows uses IOCP (I/O Completion Ports — semantically a completion model rather than a readiness model, which the Windows netpoll adapts). The choice of edge-triggered mode on Linux matters: edge-triggered epoll reports a transition (became readable) once, not the level (is readable). The runtime therefore must drain a descriptor fully and re-arm by re-attempting the syscall — which is exactly what the retry loop in internal/poll does.

Mechanical Walk-through

pollDesc — the per-descriptor state

Each pollable file descriptor has a runtime pollDesc (netpoll.go). Its heart is two word-sized fields, rg and wg — the comment calls them “2 binary semaphores… to park reader and writer goroutines respectively.” Each can hold one of four values:

  • pdNil — nothing happening.
  • pdReady — “io readiness notification is pending; a goroutine consumes the notification by changing the state to pdNil.” That is: I/O became ready before any goroutine parked.
  • pdWait — “a goroutine prepares to park on the semaphore, but not yet parked.” A transient state during the park handshake.
  • a *g pointer — “the goroutine is blocked on the semaphore.” Readiness or timeout will overwrite this and unpark the goroutine.

A pollDesc holds at most one waiting goroutine per direction — the comment is explicit: “Concurrent calls to netpollblock in the same mode are forbidden, as pollDesc can hold only a single waiting goroutine for each mode.” This is why one connection can be read by only one goroutine at a time (and written by one at a time).

Parking: from conn.Read to gopark

When a goroutine calls conn.Read, the path runs through internal/poll.FD.Read, which first attempts the non-blocking read(2) syscall. If the kernel returns data, the read is done — no poller involvement at all. If it returns EAGAIN/EWOULDBLOCK, internal/poll calls into the runtime via poll_runtime_pollWait, which calls netpollblock(pd, mode, false) (netpoll.go).

netpollblock runs a small state machine on pd.rg (for a read):

  1. if gpp.CompareAndSwap(pdReady, pdNil) { return true } — if I/O was already signalled ready, consume the notification and return immediately; no parking.
  2. if gpp.CompareAndSwap(pdNil, pdWait) { break } — otherwise transition pdNil -> pdWait, committing to park.
  3. After re-checking for errors (a deadline may have fired), it calls gopark(netpollblockcommit, unsafe.Pointer(gpp), waitReasonIOWait, traceBlockNet, 5), where gpp is &pd.rg for a read or &pd.wg for a write (netpoll.go, line 575).

gopark is the runtime primitive that deschedules the current goroutine: it transitions the G to _Gwaiting and returns the M to the scheduler to find other work. The commit callback netpollblockcommit does the final CAS pdWait -> *g (storing the goroutine pointer) and, critically, calls netpollAdjustWaiters(1) to increment netpollWaiters — a global count the scheduler reads to decide “is it worth blocking in netpoll at all?”

The goroutine is now parked. Its M is not blocked — it has gone back to findRunnable and is running other goroutines. No OS thread is consumed by this connection.

Polling: discovering readiness

The OS-specific netpoll(delta) calls epoll_wait (or kevent, etc.). It is invoked from several places in the scheduler, not by a dedicated thread:

  • Inside findRunnable (Work Stealing Scheduler): when a P is out of work, it does a non-blocking netpoll(0) to scoop up any ready goroutines, and — if there is genuinely nothing else to do anywhere — a blocking netpoll(delta) where delta is the time until the next timer.
  • From sysmon (see Sysmon System Monitor): the background monitor periodically polls so readiness is not missed when all Ps are busy.

netpoll translates each ready descriptor back to its pollDesc and calls netpollready(&toRun, pd, mode). netpollready calls netpollunblock, which moves pd.rg/pd.wg to pdReady and, if a *g was stored there, returns that goroutine. The returned goroutines are collected into a gList; netpoll’s caller transitions them to _Grunnable and places them on run queues. The next time an M runs findRunnable, it picks one up, the conn.Read retries its non-blocking syscall — now data is available — and returns.

netpollBreak — interrupting a blocking poll

There is a subtle problem: if an M is blocked in netpoll(delta) waiting for network I/O, and meanwhile a non-network goroutine becomes runnable (a timer fires early, a channel send wakes someone), that M should stop polling and go run the work. netpollBreak solves this: on Linux it writes a byte to the eventfd created at init and registered with the epoll set, causing epoll_wait to return immediately (other platforms use the analogous mechanism — a self-pipe historically, kevent user events on BSD/macOS). A netpollWakeSig atomic guards against redundant netpollBreak writes (netpoll_epoll.go, lines 17–18, 66–87). The blocked M wakes, finds the runnable goroutine, and proceeds. Without netpollBreak, newly-ready non-network work could sit unscheduled until the poll timeout expired.

Deadlines

conn.SetReadDeadline is implemented on top of the poller. A deadline arms a runtime timer; when it fires, the timer callback marks the pollDesc as timed out and uses netpollunblock to wake the parked goroutine with a pollErrTimeout error rather than data. This is why network timeouts in Go are precise and cheap — they reuse the same parking/unparking machinery and the runtime timer heap, with no extra thread.

Code: The Poller Behind a Simple Server

package main
 
import (
	"io"
	"net"
)
 
func main() {
	ln, _ := net.Listen("tcp", ":8080") // listener fd registered with the poller
	for {
		conn, err := ln.Accept() // Accept parks on the poller until a connection arrives
		if err != nil {
			return
		}
		go handle(conn) // one goroutine per connection — cheap, because idle ones park
	}
}
 
func handle(conn net.Conn) {
	defer conn.Close()
	buf := make([]byte, 4096)
	for {
		// Read first tries a non-blocking syscall. On EAGAIN the goroutine
		// parks via netpollblock; the M is freed. When epoll reports the fd
		// readable, netpollready wakes this goroutine and Read retries.
		n, err := conn.Read(buf)
		if err == io.EOF || err != nil {
			return
		}
		conn.Write(buf[:n]) // Write parks the same way on EAGAIN if the socket buffer is full
	}
}

Line-by-line on the I/O semantics: net.Listen creates a socket and registers its descriptor with the poller via netpollopen. ln.Accept() parks the accepting goroutine on the listener’s pollDesc until epoll reports an incoming connection — no thread is held while waiting. Each go handle(conn) is cheap precisely because a connection that is idle (no bytes arriving) costs only a parked goroutine — a couple of kilobytes of stack — not an OS thread. conn.Read and conn.Write both park on the poller when they would block. A server written this way can hold hundreds of thousands of connections on GOMAXPROCS threads.

# Confirm the program is poller-backed, not thread-backed:
cat /proc/$(pgrep prog)/status | grep Threads   # stays near GOMAXPROCS even with 100k conns
strace -f -e epoll_wait,epoll_ctl ./prog          # epoll_ctl on each accept; epoll_wait in the loop
GODEBUG=schedtrace=1000 ./prog                    # idle conns => goroutines parked, P count low

Failure Modes and Common Misunderstandings

“Network I/O blocks an OS thread.” False, and the most important misconception to correct. Socket/pipe I/O parks the goroutine on the poller; the M is freed. Only file I/O blocks a thread, because regular files are not pollable by epoll in the readiness sense — file reads go through blocking entersyscall (System Calls and the Scheduler). A program doing heavy file I/O grows its thread count; a program doing heavy socket I/O does not.

“More connections need more threads.” No. Connection count drives goroutine count and memory (stacks), not thread count. Thread count tracks GOMAXPROCS plus blocking file/cgo calls. A million idle connections is a memory question, not a threading question.

“Two goroutines can read the same connection concurrently.” They cannot safely — a pollDesc holds only one waiting goroutine per direction, and concurrent netpollblock calls in the same mode are forbidden. Reads of one connection must be serialised by the application; the standard pattern is one reader goroutine and one writer goroutine per connection.

“The poller is a goroutine / has its own thread.” No. There is no dedicated poller thread in the Go runtime. netpoll is called by whatever M happens to be looking for work in findRunnable, and by sysmon. Readiness discovery is opportunistic, folded into the scheduler’s idle path. This is why the poller adds essentially zero overhead when the program is busy.

Edge-triggered surprises. Because Linux epoll is armed edge-triggered (EPOLLET), a readiness event fires once per transition. If application code reads a socket only partially and stops, the runtime will not get a fresh epoll event for the remaining buffered bytes until new data arrives — but Go’s internal/poll retry loop handles this by draining until EAGAIN. Hand-rolled unsafe/syscall code that bypasses internal/poll can hit edge-triggered stalls.

Alternatives and When They Apply

The alternatives: a blocking thread per connection (the classic Apache prefork model — simple, but caps out in the low thousands of connections); a hand-written event loop (epoll/libevent/libuv, the Node.js and nginx model — scales to millions but forces callback-style code); and io_uring (a newer Linux completion-based interface, potentially fewer syscalls than epoll). Go’s integrated poller occupies a deliberate middle ground: it is an event loop internally, but it is fused with the goroutine scheduler so application code stays synchronous. Go does not use io_uring for its general netpoller; epoll remains the Linux backend as of Go 1.26. A directory listing of src/runtime on the Go master branch shows exactly these netpoll backends — netpoll_epoll.go (Linux), netpoll_kqueue.go (BSD/macOS), netpoll_solaris.go (event ports), netpoll_windows.go (IOCP), plus netpoll_aix.go, netpoll_wasip1.go, netpoll_fake.go, and netpoll_stub.go — and no netpoll_iouring.go (src/runtime listing). io_uring adoption has long been discussed in the Go issue tracker and several third-party libraries exist (e.g. cloudwego/netpoll, godzie44/go-uring), but the standard runtime poller has not adopted it.

Production Notes

The poller is invisible and needs no tuning, but it shapes how Go servers are operated and debugged. Capacity planning for a network server is a memory exercise (goroutine stacks × connections) and a file-descriptor exercise (ulimit -n), not a thread exercise. If a network server’s thread count climbs, the cause is not connections — it is concurrent blocking file/DNS/cgo calls (DNS resolution via cgo’s getaddrinfo is a frequent offender; the pure-Go resolver avoids it). Diagnose with the /proc/<pid>/status thread count, the runtime/pprof threadcreate profile, and the execution tracer’s network-blocking events (The Go Execution Tracer). Deadlines (SetReadDeadline/SetWriteDeadline) are the correct, cheap way to bound network waits — they reuse the poller’s parking machinery and the runtime timer heap. The poller is also why Go scales gracefully under connection floods where a thread-per-connection server would collapse.

See Also

Graph View

Backlinks