io_uring as a Syscall Batching Mechanism

io_uring is Linux’s modern asynchronous I/O interface (merged in kernel 5.1, May 2019, by Jens Axboe — independently verified by existence-check: fs/io_uring.c returns HTTP 404 at the v5.0 tag and HTTP 200 at v5.1), and the single most direct architectural answer to the question “why are system calls expensive?”. A conventional program performs one I/O operation per system call: each read(), write(), send(), or recv() traps into the kernel, pays the full mode-switch cost (privilege transition, register save/restore, cache and TLB disturbance, plus the Spectre/Meltdown mitigations layered on after 2018), and traps back out. io_uring breaks that one-to-one coupling. The kernel and the application share two ring buffers in memory — a submission queue (SQ) the application fills with work and a completion queue (CQ) the kernel fills with results — so an application can queue thousands of operations and then submit them all with a single io_uring_enter(2) call, or, with the IORING_SETUP_SQPOLL option, with zero system calls in steady state because a dedicated kernel thread polls the queue. This note treats io_uring through the syscall-batching lens: why amortizing the fixed trap cost across many operations is a categorical win, and the three syscalls (io_uring_setup, io_uring_enter, io_uring_register) that set it up and drive it. The ring mechanics themselves — the memory layout, indices, and barriers — are the subject of its sibling, The io_uring Submission and Completion Queues.

Uncertain

Version pinning: all opcode/flag claims here are verified against the uapi header at the Linux v6.12 tag (include/uapi/linux/io_uring.h) and the v6.12 / v6.18 LTS sysctl/kernel.rst. io_uring’s opcode surface grows nearly every release; “the current set of operations” is a point-in-time claim as of 6.12 LTS / 6.18 LTS (2025-11-30). To resolve a specific opcode’s availability, check the uapi header at the exact tag you target. uncertain

---

Mental Model

The way to think about io_uring is producer/consumer queues replacing call/return. A normal syscall is a function call across the privilege boundary: you call, you block (conceptually), the kernel returns, and only then can you issue the next request. io_uring inverts this into a mailbox: you drop letters (submission queue entries, SQEs) into an outbox, the kernel picks them up and works on them asynchronously and out of order, and it drops results (completion queue entries, CQEs) into an inbox you read at your leisure. The boundary crossing — the expensive part — happens only when you ring the doorbell (io_uring_enter), and a single doorbell ring can announce a hundred letters at once. With IORING_SETUP_SQPOLL you remove even the doorbell: a kernel thread sits watching the outbox and grabs letters as they appear.

flowchart LR
  subgraph US["Userspace"]
    APP["Application"]
    SQ["Submission Queue<br/>(SQEs: read, write,<br/>send, recv, ...)"]
    CQ["Completion Queue<br/>(CQEs: result + user_data)"]
    APP -->|"fill many SQEs<br/>(no syscall)"| SQ
    CQ -->|"reap results<br/>(no syscall)"| APP
  end
  subgraph K["Kernel"]
    direction TB
    ENTER["io_uring_enter()<br/>ONE syscall submits N ops"]
    SQPOLL["SQPOLL kernel thread<br/>ZERO syscalls steady state"]
    WORK["block / net / fs<br/>backends do the IO"]
  end
  SQ -->|"doorbell"| ENTER
  SQ -.->|"polled"| SQPOLL
  ENTER --> WORK
  SQPOLL --> WORK
  WORK -->|"post completions"| CQ

How io_uring batches the boundary crossing. What it shows: the application fills the submission queue and reads the completion queue entirely in shared memory with no system call; the boundary is crossed only at the doorbell (io_uring_enter), which can carry N operations at once, or — on the SQPOLL path — not at all, because a kernel thread polls the queue. The insight: the fixed per-syscall cost is paid once per batch (or never), not once per operation, so the per-operation overhead approaches zero as the batch grows.

Readiness versus Completion — the One Distinction That Explains Everything

Before going further it is worth fixing the single conceptual axis that separates io_uring from every interface it competes with, because almost every design consequence falls out of it.

• A readiness interface tells you when you may act. epoll, poll, select, and BSD kqueue all answer the question “which of my descriptors would not block if I called read on them right now?” You then still have to make the call. The kernel’s job is notification; the I/O is yours.
• A completion interface tells you that the act is done. io_uring, Windows I/O Completion Ports, and Linux’s older aio all answer “which of the operations I asked for have finished, and what did they return?” You hand over the whole operation — opcode, descriptor, buffer, length — and the kernel performs it and posts a result.

Everything else follows. Under readiness you pay one syscall for the notification (epoll_wait) plus one syscall per ready descriptor to do the actual I/O, so a tick that finds 100 ready sockets costs 101 boundary crossings. Under completion you pay one syscall to submit and collect, so the same 100 operations cost one crossing — or zero under SQPOLL. Readiness also only works for descriptors that have a meaningful “would block” state: sockets, pipes, terminals, eventfd, timerfd, inotify. A regular file on a disk is always “ready” in the readiness model even when reading it will stall for milliseconds on a page fault, which is why epoll_ctl(EPOLL_CTL_ADD) on a regular file is rejected outright with EPERM (verified in do_epoll_ctl, fs/eventpoll.c at v6.12: error = -EPERM; if (!file_can_poll(fd_file(tf))) goto error_tgt_fput;). Completion has no such blind spot — an IORING_OP_READ against a regular file is exactly as asynchronous as one against a socket. That gap is a large part of why io_uring exists at all.

sequenceDiagram
    autonumber
    participant A as Application
    participant K as Kernel
    Note over A,K: READINESS model (epoll) — 1 + N syscalls per tick
    A->>K: epoll_wait()          [syscall 1]
    K-->>A: 3 fds are readable
    A->>K: read(fd_a)            [syscall 2]
    A->>K: read(fd_b)            [syscall 3]
    A->>K: read(fd_c)            [syscall 4]
    Note over A,K: COMPLETION model (io_uring) — 1 syscall, or 0 with SQPOLL
    A->>A: fill 3 SQEs in shared memory (no syscall)
    A->>K: io_uring_enter(to_submit=3, min_complete=3)  [syscall 1]
    K-->>A: 3 CQEs written straight into the shared CQ ring
    A->>A: read results from shared memory (no syscall)

Readiness versus completion, counted in boundary crossings. What it shows: the same three reads cost four system calls under epoll (one wait plus one read per ready descriptor) and exactly one under io_uring, because the read itself has been moved into the batch rather than left outside it. The insight: epoll batches only the notification; io_uring batches the operations. That is why epoll’s syscall count grows with the number of ready descriptors while io_uring’s does not, and it is also why epoll can never help with regular-file I/O — there is no “ready” state to report. See epoll and Scalable Readiness Notification for the readiness side of this contrast in full.

---

Why One Syscall Per Operation Is a Scaling Ceiling

The motivation is laid out plainly in Jens Axboe’s design document Efficient IO with io_uring (kernel.dk/io_uring.pdf). The traditional synchronous calls — read(2), write(2), pread/pwrite, preadv/pwritev, preadv2/pwritev2 — all “return when the data is ready (or written),” meaning the calling thread blocks across the syscall. To get asynchrony, applications historically either spun up private I/O thread pools (one blocked thread per outstanding operation — expensive and unscalable) or used Linux’s native AIO (io_setup/io_submit/io_getevents), which Axboe describes as suffering “a number of limitations”: it only does truly async I/O for O_DIRECT (cache-bypassing) access, its submission can still block “if meta data is required to perform IO,” and — critically for this note — “IO always requires at least two system calls (submit + wait-for-completion), which in these post spectre/meltdown days is a serious slowdown.”

That last clause is the whole story. The companion note Why System Calls Are Expensive dissects the fixed cost of a single trap: the CPU privilege transition (syscall/sysret), the register save/restore into a pt_regs frame, the generic entry/exit bookkeeping, and — after 2018 — the speculative-execution mitigations (KPTI page-table switches, retpolines, IBRS/STIBP barriers) that inflated the cost of every boundary crossing by hundreds of cycles. When that fixed cost is paid once per I/O and you are doing millions of small I/Os per second on a fast NVMe device or a busy network socket, the syscall overhead alone can dominate, and “lack of performance that you can extract out of a single core” becomes the bottleneck, as Axboe notes for “devices that are capable of both sub-10usec latencies and very high IOPS.”

io_uring’s answer is amortization. If submitting N operations costs one io_uring_enter instead of N separate syscalls, the per-operation fixed overhead falls by a factor of N. With a deep enough queue the syscall cost per operation is negligible; with SQPOLL it disappears entirely. This is the same arithmetic that motivates the simpler batched calls — writev (one syscall, many buffer segments for one fd) and recvmmsg (one syscall, many datagrams) — but io_uring generalizes it: heterogeneous operations across many file descriptors, fully asynchronous, in a single batch. Where sendmmsg batches one operation type on one socket, io_uring batches reads, writes, accepts, connects, fsyncs, timeouts, and dozens of other opcodes across arbitrary descriptors in one ring.

The Arithmetic, Made Concrete

Let c be the fixed cost of one boundary crossing and w the useful work of one operation. Under one-syscall-per-op the cost of N operations is N × (c + w). Under io_uring it is c + N × w, and under SQPOLL simply N × w. The fraction of time spent on overhead is therefore c / (c + w) in the first case and c / (c + N × w) in the second — it decays as 1/N. This is why io_uring’s advantage is not a constant factor but grows with batch depth, and equally why it disappears at N = 1, where c + 1 × w is the same as 1 × (c + w) and io_uring’s extra ring bookkeeping makes it strictly worse.

Batch depth N	Syscalls, one-per-op	Syscalls, io_uring	Syscalls, io_uring + SQPOLL	Crossings saved
1	1	1	0	0 (or 1)
8	8	1	0	87.5% (100%)
64	64	1	0	98.4% (100%)
512	512	1	0	99.8% (100%)
32,768 (max SQ)	32,768	1	0	99.997% (100%)

Boundary crossings per batch of N operations. What it shows: the syscall column for io_uring is the constant 1 regardless of N, and 0 under SQPOLL; the saving is 1 − 1/N. The insight: the curve is steep early and then flat — going from 1 to 8 operations per call already removes 87% of the crossings, while going from 512 to 32,768 buys almost nothing more. Batching pays off long before you need a deep queue, which is why even modest opportunistic batching in an event loop is worth doing. The 32,768 ceiling is IORING_MAX_ENTRIES in io_uring/io_uring.c at v6.12 — note this corrects Axboe’s 2019 design document, which states a 1..4096 range that was accurate only for the original 5.1 implementation.

Axboe’s Measured Numbers

Adjectives are cheap; the design document gives figures, and they are worth quoting precisely because they bound what one should expect (Efficient IO with io_uring, §9):

Measurement	Figure	What it isolates
Peak per-core 4 KiB random read, io_uring with polling	~1,700K IOPS	Best case: batching and IOPOLL, no interrupts, no syscall per op
Same workload, io_uring without polling	~1,200K IOPS	Batching alone — still ~2× aio
Same workload, Linux native aio	608K IOPS (a “performance cliff”)	The two-syscall-per-IO ceiling
IORING_OP_NOP round-trip, raw interface	12M msgs/s (laptop) to 20M msgs/s (test box)	The interface itself with zero I/O work — “mostly bound by the number of system calls that have to be performed”
aio bytes copied per operation	64 + 8 submit, 32 complete = 104 bytes	“104 bytes of memory copy, for IO that’s supposedly zero copy”

Reported throughput across the three mechanisms. What it shows: the ~2× gap between io_uring-without-polling and aio at 608K is attributable almost entirely to syscall count, since both do the same block-layer work; the further jump to 1.7M comes from adding IOPOLL, which removes the completion interrupt as well. The insight: the NOP figure is the most diagnostic one — 12–20 million no-op operations per second is the interface’s own ceiling, so for any real workload io_uring is no longer the bottleneck, which is exactly the design goal Axboe states. Caveat: these are 2019-era numbers on the author’s hardware, and the document itself says they “are a bit outdated” and “don’t carry a lot of absolute meaning”; treat the ratios, not the absolutes, as the transferable fact.

Uncertain

Verify: the IOPS figures above are single-machine measurements published by io_uring’s author in a design document, not an independently reproduced benchmark, and the document explicitly disclaims their absolute accuracy. Reason: no peer-reviewed or third-party replication was located during this research; kernel.dk was unreachable directly (see the source note below) and the PDF was read from a Wayback Machine snapshot. To resolve: reproduce with fio’s io_uring engine against a known NVMe device, or cite a published storage-vendor benchmark that states its hardware. uncertain

Uncertain

Verify: https://kernel.dk/io_uring.pdf — the canonical home of Axboe’s design document — could not be fetched during this research. Reason: curl fails TLS verification with SSL: no alternative certificate subject name matches target hostname 'kernel.dk', i.e. the served certificate does not cover that hostname. The document was therefore read from the Internet Archive snapshot web.archive.org/web/20241228075527id_/https://kernel.dk/io_uring.pdf (snapshot dated 2024-12-28), whose contents were verified to match the expected title and section structure. To resolve: retry the canonical URL from a browser, or fetch it from a mirror once the certificate is fixed. uncertain

---

The Ring Geometry a Batch Moves Through

Batching is not an abstract policy — it is a concrete consequence of pointer geometry. Two ring buffers live in memory mapped into both the application’s and the kernel’s address space, and each is governed by a pair of free-running 32-bit counters, a head and a tail. Which side owns which counter is the entire protocol, and once you can see that, the batching property becomes obvious rather than magical. (The full mmap layout, the barrier discipline, and the field-by-field struct dissection belong to the sibling note The io_uring Submission and Completion Queues; what follows is the minimum geometry needed to see why one doorbell can carry N operations.)

Ring	head advanced by	tail advanced by	Meaning of tail - head
Submission queue (SQ)	the kernel, once it has consumed entries	the application, once it has filled entries	how many operations are queued but not yet consumed
Completion queue (CQ)	the application, once it has read results	the kernel, once it has posted results	how many results are waiting to be reaped

Index ownership. What it shows: on each ring exactly one side writes each counter — single-producer/single-consumer in both directions. The insight: because the application owns the SQ tail, it can advance that tail by 1 or by 500 with the same single store; the kernel learns about all 500 the instant it next reads the tail. Nothing in the protocol charges per entry, which is precisely what makes the batch free.

A second piece of geometry matters: the SQ ring does not contain the submission queue entries (SQEs) themselves. It contains an indirection array — array[], a plain array of 32-bit indices — and the actual 64-byte struct io_uring_sqe records live in a separately mapped region. Axboe’s stated reason is that this “allows an application to easily support having independent [SQE] rings” and to embed SQEs inside its own data structures, submitting them in whatever order it likes (design doc §4.2). The completion ring has no such indirection: its 16-byte CQEs sit inline in the ring.

flowchart LR
  subgraph SQR["SQ ring (mmap at IORING_OFF_SQ_RING)"]
    direction TB
    IDX["array[]<br/>[0]=3  [1]=0  [2]=7  [3]=1"]
    CNT["head (kernel writes)<br/>tail (app writes)<br/>ring_mask, ring_entries"]
  end
  subgraph SQEA["SQE array (mmap at IORING_OFF_SQES) — 64 bytes each"]
    direction TB
    E0["sqe[0]"]
    E1["sqe[1]"]
    E3["sqe[3]"]
    E7["sqe[7]"]
  end
  IDX -->|"array[0] = 3"| E3
  IDX -->|"array[1] = 0"| E0
  IDX -->|"array[2] = 7"| E7
  IDX -->|"array[3] = 1"| E1

The submission-side indirection, drawn as pointers. What it shows: slot i of the ring does not hold an SQE; it holds an index into the SQE array, so ring order and SQE storage order are decoupled. The insight: an application can keep a long-lived sqe embedded in each connection object, never copy it, and submit connections in arbitrary order simply by writing their indices into array[]. Copying 64 bytes per submission — the thing Axboe criticises Linux aio for — is avoided entirely.

The Same Ring in Three Successive States

The clearest way to internalise the protocol is to watch one batch of four operations move through it. Assume a ring with ring_entries = 8, hence ring_mask = 7; the counters are free-running and the slot is always counter & ring_mask.

State 0 — idle. Nothing queued, nothing outstanding. head == tail on both rings.

flowchart LR
  subgraph S0SQ["SQ: head=12, tail=12 (empty)"]
    direction LR
    A0["slot 4<br/>(12 mod 8)<br/>H,T"]:::hd
    A1["slot 5"]:::free
    A2["slot 6"]:::free
    A3["slot 7"]:::free
  end
  subgraph S0CQ["CQ: head=9, tail=9 (empty)"]
    direction LR
    B0["slot 1<br/>(9 mod 8)<br/>H,T"]:::hd
    B1["slot 2"]:::free
    B2["slot 3"]:::free
    B3["slot 4"]:::free
  end
  classDef hd fill:#f9d,stroke:#333
  classDef free fill:#eee,stroke:#999
  classDef full fill:#9df,stroke:#333

State 1 — the application has filled four SQEs and published the tail. This is pure memory traffic: four 64-byte writes into the SQE array, four 32-bit writes into array[], and one release-store advancing tail from 12 to 16. No system call has happened yet. The kernel’s head is still 12, so from the kernel’s point of view four entries are pending.

flowchart LR
  subgraph S1SQ["SQ: head=12, tail=16 → 4 pending (ONE store published all four)"]
    direction LR
    A0["slot 4<br/>READ fd=5<br/>HEAD"]:::full
    A1["slot 5<br/>READ fd=6"]:::full
    A2["slot 6<br/>WRITE fd=7"]:::full
    A3["slot 7<br/>FSYNC fd=7"]:::full
    A4["slot 0<br/>(16 mod 8)<br/>TAIL"]:::hd
  end
  subgraph S1CQ["CQ: head=9, tail=9 (still empty)"]
    direction LR
    B0["slot 1<br/>H,T"]:::hd
  end
  classDef hd fill:#f9d,stroke:#333
  classDef free fill:#eee,stroke:#999
  classDef full fill:#9df,stroke:#333

State 2 — after one io_uring_enter(fd, 4, 4, IORING_ENTER_GETEVENTS, NULL). The kernel consumed all four SQEs, ran them, posted four CQEs, and advanced both counters it owns. Note in the v6.12 source that io_submit_sqes() loops over the entries and then calls io_commit_sqring(ctx) once, after the loop — a single smp_store_release(&rings->sq.head, ctx->cached_sq_head). The SQ head is not advanced per SQE; it is advanced per batch.

flowchart LR
  subgraph S2SQ["SQ: head=16, tail=16 (drained — kernel advanced head ONCE)"]
    direction LR
    A0["slot 0<br/>(16 mod 8)<br/>H,T"]:::hd
    A1["slot 1"]:::free
  end
  subgraph S2CQ["CQ: head=9, tail=13 → 4 results waiting"]
    direction LR
    B0["slot 1<br/>res=4096<br/>HEAD"]:::full
    B1["slot 2<br/>res=4096"]:::full
    B2["slot 3<br/>res=512"]:::full
    B3["slot 4<br/>res=0"]:::full
    B4["slot 5<br/>(13 mod 8)<br/>TAIL"]:::hd
  end
  classDef hd fill:#f9d,stroke:#333
  classDef free fill:#eee,stroke:#999
  classDef full fill:#9df,stroke:#333

One batch of four operations, drawn in three successive states. What it shows: between State 0 and State 1 the application did all its work with plain stores and published it by advancing one counter; between State 1 and State 2 the kernel consumed the whole batch and advanced one counter on each ring. The insight: at no point does the protocol cost anything per operation — the per-batch costs are a single release-store on each side plus one io_uring_enter. The free-running counters also explain the wrap: tail went 12 → 16 and the slot index wrapped from 7 back to 0 via & ring_mask, with no special-casing, because the counters are allowed to run past ring_entries and only the masked low bits select a slot. Reaping the four results needs no syscall at all — the application reads CQ.tail, walks slots 1..4, and advances CQ.head to 13, entirely in shared memory.

The Two Entry Structures

The submission queue entry is 64 bytes (128 with IORING_SETUP_SQE128) and the completion queue entry is 16 (32 with IORING_SETUP_CQE32). The CQE is small enough to draw exactly, and it is the more instructive of the two because it explains how completions are matched to submissions out of order:

packet-beta
0-63: "user_data (__u64) — the opaque cookie copied verbatim from sqe.user_data"
64-95: "res (__s32) — return value, or a negative errno"
96-127: "flags (__u32) — IORING_CQE_F_MORE, F_BUFFER, F_SOCK_NONEMPTY, F_NOTIF"

The 16-byte struct io_uring_cqe, verified against the v6.12 uapi header. What it shows: a completion carries no descriptor, no opcode, and no offset — only the cookie you supplied, the result, and flags. The insight: because completions arrive out of order, the kernel needs a way to say which request finished, and user_data is that way; there is deliberately nothing else. In practice you store a pointer to your own request object there, so matching a CQE to its originating work is a single pointer dereference rather than a lookup. res follows the syscall convention exactly: ≥ 0 is the byte count or success value, negative is -errno — so a failed IORING_OP_READ yields res = -EBADF rather than setting errno, since no libc wrapper was involved.

The SQE has too many overlapping unions to draw bit-accurately (the flags word alone is a 23-way union at v6.12), so it is clearest as a byte-offset table. Only the fields that matter to batching are commented here; the full dissection lives in The io_uring Submission and Completion Queues.

Offset	Size	Field	Why it matters to batching
0	1	opcode	Each SQE in a batch may be a different operation — this is what lets one io_uring_enter carry reads, writes, and an fsync together
1	1	flags	IOSQE_*: FIXED_FILE, IO_LINK, IO_HARDLINK, IO_DRAIN, ASYNC, BUFFER_SELECT, CQE_SKIP_SUCCESS — the ordering and resource controls inside a batch
2	2	ioprio	Per-request I/O priority
4	4	fd	Descriptor, or an index into the registered-file table when IOSQE_FIXED_FILE is set
8	8	off / addr2 / cmd_op	Offset — so many SQEs can target one file at different offsets without any shared seek position
16	8	addr / splice_off_in	Buffer pointer or iovec array
24	4	len	Byte count or iovec count
28	4	rw_flags / *_flags (union)	Per-opcode flags
32	8	user_data	The cookie echoed back in the CQE — the only link between an SQE and its completion
40	2	buf_index / buf_group	Selects a registered buffer, or a provided-buffer group
42	2	personality	Credentials to run the operation under
44	4	splice_fd_in / file_index	Splice source, or destination slot for direct-descriptor opcodes
48	16	addr3 + padding / optval / cmd[]	Third address, or the start of 80 bytes of command payload under SQE128

The 64-byte struct io_uring_sqe by byte offset, from the v6.12 uapi header. What it shows: an SQE is a complete, self-describing operation — opcode, target, offset, buffer, length, flags — with no dependence on any per-descriptor kernel state such as a file position. The insight: self-description is what makes out-of-order, heterogeneous batching possible at all. A readv batch shares one fd and one implicit file offset; a batch of SQEs shares nothing, so the kernel is free to dispatch them concurrently to entirely different subsystems. A mermaid packet-beta diagram was not used for this struct because most of its 64 bytes are occupied by mutually exclusive unions whose meaning depends on opcode; a bit-range drawing would assert a single interpretation that is false for most opcodes.

---

The Three Syscalls

io_uring is, from the kernel’s ABI perspective, just three new system calls. Everything else is shared-memory manipulation that crosses no boundary.

io_uring_setup(2) — create the rings

int io_uring_setup(u32 entries, struct io_uring_params *params);

entries requests the submission-queue depth, rounded up to a power of two (p->sq_entries = roundup_pow_of_two(entries)). At v6.12 the ceiling is IORING_MAX_ENTRIES = 32768, and the completion ring’s ceiling is IORING_MAX_CQ_ENTRIES = 2 * IORING_MAX_ENTRIES = 65536 — both verified in io_uring/io_uring.c. Exceeding the SQ ceiling returns -EINVAL unless IORING_SETUP_CLAMP is set, in which case the request is silently clamped instead. This corrects the design document, which states a 1..4096 range — accurate for the original 5.1 implementation, stale since. The kernel allocates the SQ and CQ ring buffers, returns a file descriptor referring to the io_uring instance, and fills in params — most importantly the sq_off and cq_off offset structures the application uses to mmap(2) the shared rings into its address space (the full layout is covered in The io_uring Submission and Completion Queues). The flags field of struct io_uring_params selects setup options; the batching-relevant ones:

• IORING_SETUP_SQPOLL (1U << 1 in the v6.12 uapi header) — spawn a kernel thread that polls the SQ, so the application submits work without any syscall (detailed below).
• IORING_SETUP_IOPOLL — busy-poll the device for completions instead of waiting for an interrupt; requires O_DIRECT and device polling support, trading CPU for the lowest possible completion latency.
• IORING_SETUP_SQ_AFF — pin the SQPOLL thread to the CPU named in sq_thread_cpu (only meaningful with SQPOLL).
• IORING_SETUP_CQSIZE — size the completion queue explicitly via cq_entries. Without it the kernel picks p->cq_entries = 2 * p->sq_entries, and the source comment explains why: “It’s possible for the application to drive a higher depth than the size of the SQ ring, since the sqes are only used at submission time. This allows for some flexibility in overcommitting a bit.” With it, cq_entries is rounded up to a power of two and then checked with if (p->cq_entries < p->sq_entries) return -EINVAL — so the CQ must be at least as large as the SQ, not strictly larger as the man page’s wording suggests (v6.12 io_uring_create).
• IORING_SETUP_SINGLE_ISSUER (1U << 12) — promise the kernel that only one task ever submits to this ring, enabling locking optimizations; the kernel enforces it, rejecting submissions from other tasks with -EEXIST.
• IORING_SETUP_DEFER_TASKRUN (1U << 13) — defer the kernel’s completion-processing work until the application next calls io_uring_enter with IORING_ENTER_GETEVENTS, batching that work too; requires IORING_SETUP_SINGLE_ISSUER.

The last two flags are the modern (post-5.18-era) high-performance configuration: a single-issuer ring with deferred task-run minimizes both contention and the random interruptions completions would otherwise cause. They are verified present in the v6.12 uapi header.

io_uring_enter(2) — the doorbell

int io_uring_enter(unsigned int fd, unsigned int to_submit,
                   unsigned int min_complete, unsigned int flags,
                   sigset_t *sig);

This is the one syscall that does the batching. to_submit tells the kernel how many freshly-queued SQEs to consume; a single call submits all of them. If flags contains IORING_ENTER_GETEVENTS (1U << 0), the call also waits for min_complete completions before returning — so a single io_uring_enter can submit N operations and wait for M results in one boundary crossing, which the design doc calls out as the key efficiency win: “the application can both submit and wait for request completions with a single system call.” Returning, it reports the number of SQEs consumed. Crucially, when you only want to reap completions you do not even need this call — the kernel updates the CQ ring tail directly in shared memory, so the application can read completions by polling the CQ tail with no syscall at all (see The io_uring Submission and Completion Queues).

The v6.12 source makes the batching explicit. io_uring_enter reaches io_submit_sqes(ctx, to_submit), which loops do { ... io_submit_sqe(ctx, req, sqe); } while (--left); and only then calls io_commit_sqring(ctx) — one smp_store_release of the SQ head for the entire batch. On the wait side, the same call falls through to io_cqring_wait(ctx, min_complete, ...) (or io_iopoll_check on an IOPOLL ring). Submit and wait are two halves of one syscall body, not two syscalls.

sequenceDiagram
    autonumber
    participant App as Application
    participant SQ as SQ ring<br/>(shared memory)
    participant Sys as io_uring_enter
    participant Sub as io_submit_sqes
    participant BE as Backend<br/>(block / net / fs)
    participant CQ as CQ ring<br/>(shared memory)

    Note over App,SQ: Phase 1 — fill the batch. No boundary crossing.
    loop N times
        App->>SQ: write the SQE, then set array slot tail mod 8 to its index
    end
    App->>SQ: smp_store_release(sq.tail, tail + N)

    Note over App,Sys: Phase 2 — ONE boundary crossing
    App->>Sys: io_uring_enter(fd, to_submit=N,<br/>min_complete=M, IORING_ENTER_GETEVENTS)
    Sys->>Sub: io_submit_sqes(ctx, N)
    loop N times
        Sub->>BE: io_submit_sqe → io_queue_sqe → dispatch
    end
    Sub->>SQ: io_commit_sqring — ONE store_release of sq.head
    Note right of Sub: head advanced once per BATCH,<br/>not once per SQE
    Sys->>Sys: io_cqring_wait(ctx, M, ...) — sleep until M results

    Note over BE,CQ: Phase 3 — completions land asynchronously, out of order
    BE-->>CQ: post cqe{user_data, res, flags} — publish cq.tail
    BE-->>CQ: post cqe{...}
    CQ-->>Sys: M completions available → wake
    Sys-->>App: return (number of SQEs consumed)

    Note over App,CQ: Phase 4 — reap. No boundary crossing.
    App->>CQ: read cq.tail, walk cqes[], advance cq.head

The full life of one batch through io_uring_enter. What it shows: four phases, of which only phase 2 crosses the privilege boundary. Phases 1 and 4 are ordinary loads and stores against mapped memory; phase 3 happens inside the kernel with the application asleep. The insight: the number of boundary crossings is one, independent of N and M, and the IORING_ENTER_GETEVENTS flag is what fuses “submit” and “wait” into that single crossing — the property Axboe singles out as the key efficiency win, and the thing Linux aio could never do because it required a separate io_getevents. Note also that completions are posted by whichever backend finished, in completion order, so CQE k need not correspond to SQE k; user_data is the only correspondence.

io_uring_register(2) — pre-register hot resources

int io_uring_register(unsigned int fd, unsigned int opcode,
                      void *arg, unsigned int nr_args);

This third call does not submit I/O; it pre-registers resources to strip per-operation overhead off the fast path, an orthogonal optimization to batching but part of the same “stop paying repeated fixed costs” philosophy:

• IORING_REGISTER_BUFFERS (fixed buffers): the kernel pins the buffer pages and builds durable kernel mappings once. Subsequent I/O uses IORING_OP_READ_FIXED / IORING_OP_WRITE_FIXED with buf_index selecting the registered buffer, so the kernel skips the per-I/O page pinning and address translation entirely.
• IORING_REGISTER_FILES (fixed files): pre-register file descriptors so the kernel does not take and drop a reference on the file table for every operation — a real win in multithreaded programs with a shared file table. Operations then set the IOSQE_FIXED_FILE flag and put the registered index in the fd field.
• IORING_REGISTER_RING_FDS (registered ring fds, added in Linux 5.18 — verified absent from the v5.17 uapi header and present at v5.18): register the ring fd itself so io_uring_enter can be called with IORING_ENTER_REGISTERED_RING (1U << 4 in v6.12), replacing the fget(fd) descriptor-table lookup at the top of the syscall with an index into a small per-task array. The v6.12 dispatch shows both paths side by side:

  if (flags & IORING_ENTER_REGISTERED_RING) {
      if (unlikely(!tctx || fd >= IO_RINGFD_REG_MAX))   /* IO_RINGFD_REG_MAX == 16 */
          return -EINVAL;
      fd = array_index_nospec(fd, IO_RINGFD_REG_MAX);
      file = tctx->registered_rings[fd];                /* array index — no refcount */
  } else {
      file = fget(fd);                                  /* full fd-table lookup + get */
      ...
  }

  The cap of 16 is IO_RINGFD_REG_MAX in include/linux/io_uring_types.h, matching the man page.

Why Registration Is a Different Kind of Saving

Batching removes boundary crossings; registration removes per-operation kernel bookkeeping that happens on the inside. They compose, and they attack different terms in the cost model. Concretely, a plain buffered read through io_uring still has to (a) look up fd in the file table and take a reference, (b) pin the user pages the buffer lives on, and (c) build a scatter-gather mapping for them — every single time. Registration does each of these once, up front.

Registration	Opcode	Per-op work removed	v6.12 limit	Where the operation opts in
Fixed buffers	IORING_REGISTER_BUFFERS (0)	Page pinning and address translation for the I/O buffer	IORING_MAX_REG_BUFFERS = 1 << 14 = 16,384	IORING_OP_READ_FIXED / WRITE_FIXED, with sqe->buf_index
Fixed files	IORING_REGISTER_FILES (2)	fget/fput on the shared file table per operation	IORING_MAX_FIXED_FILES = 1 << 20 = 1,048,576	IOSQE_FIXED_FILE in sqe->flags, with the registered index in sqe->fd
Registered ring fd	IORING_REGISTER_RING_FDS (20)	fget on the ring’s own descriptor at each io_uring_enter	IO_RINGFD_REG_MAX = 16	IORING_ENTER_REGISTERED_RING in the enter flags

The three registrations, their limits, and what each removes. What it shows: every row eliminates a fixed cost that would otherwise recur once per operation (or once per doorbell, for the last row). Limits are from io_uring/rsrc.c and io_uring_types.h at v6.12. The insight: the fixed-file win is the one people underestimate. fget/fput take a reference on a structure shared by every thread in the process, so on a busy multithreaded server they generate cache-line ping-pong on a hot refcount — the cost is not the instruction count but the contention, and it grows with your thread count while the batching win does not. Axboe’s document makes exactly this point: for a threaded application the file-reference atomics “can be a noticeable slowdown for high IOPS workloads.”

flowchart TD
    START["An IORING_OP_READ is issued"] --> Q1{"IOSQE_FIXED_FILE<br/>set?"}
    Q1 -->|no| F1["fget(sqe.fd)<br/>atomic refcount on the<br/>shared file table"]:::cost
    Q1 -->|yes| F2["registered_files[sqe.fd]<br/>plain array index"]:::fast
    F1 --> Q2{"opcode is<br/>READ_FIXED?"}
    F2 --> Q2
    Q2 -->|no| B1["pin user pages<br/>+ build sg mapping<br/>EVERY operation"]:::cost
    Q2 -->|yes| B2["reuse the mapping built<br/>at REGISTER_BUFFERS time<br/>via sqe.buf_index"]:::fast
    B1 --> ISSUE["issue to the backend"]
    B2 --> ISSUE
    ISSUE --> Q3{"IORING_SETUP_IOPOLL?"}
    Q3 -->|no| C1["completion arrives<br/>via device interrupt"]:::cost
    Q3 -->|yes| C2["io_iopoll_check spins<br/>on the device queue<br/>no interrupt"]:::fast
    C1 --> DONE["post CQE"]
    C2 --> DONE
    classDef cost fill:#fdd,stroke:#c33
    classDef fast fill:#dfd,stroke:#3a3

The three independent fast-path decisions inside one operation. What it shows: each diamond is an orthogonal opt-in — fixed files, fixed buffers, and polled completions can be enabled in any combination, and each removes a distinct recurring cost (red boxes) in favour of pre-computed state or busy-waiting (green boxes). The insight: io_uring’s performance story is not one switch but a stack of them, and batching is only the outermost layer. An application that batches heavily but registers nothing still pays a refcount and a page-pinning pass per operation; that is why Axboe’s 1.7M-IOPS figure required polling on top of batching, not batching alone.

Uncertain

Verify: the claim that fixed-file registration’s benefit comes primarily from cache-line contention on a shared refcount rather than from raw instruction count. Reason: the design document states the effect qualitatively (“noticeable slowdown for high IOPS workloads” for threaded applications) but publishes no per-operation cycle breakdown, and no profile isolating the fget/fput cost was located during this research. To resolve: profile a multithreaded fio run with and without --fixedbufs/registered files under perf c2c, which reports cache-line contention directly. uncertain

---

The Three Modes and Their Very Different Cost Profiles

io_uring is not one thing with one performance profile. The flags passed to io_uring_setup select among three operating modes that differ enormously in syscall count, CPU consumption, latency, and applicability. Choosing wrongly is the most common way to be disappointed by io_uring, so it is worth laying them out side by side before dissecting each.

	Plain (no polling flags)	IORING_SETUP_SQPOLL	IORING_SETUP_IOPOLL
What is polled	nothing	the submission queue, by a kernel thread	the device completion queue, by the submitting task
Syscalls to submit N ops	1 (io_uring_enter)	0 while the poller is awake	1
Syscalls to reap completions	0 (read the CQ in shared memory) or 1 if you want to block	0	1 — io_uring_enter(..., GETEVENTS) drives io_iopoll_check
How a completion is noticed	device interrupt → softirq → CQE posted	device interrupt → softirq → CQE posted	the task spins on the block device’s poll queue; the interrupt is not used
Extra CPU burned	none	~1 core while busy, decaying to 0 after sq_thread_idle	one core spins inside the syscall until min_complete results arrive
Works with	everything	everything	only O_DIRECT on a device whose driver has an ->iopoll method
Best for	general async I/O; the sane default	sustained high-rate submission where a dedicated core is affordable	sub-10 µs NVMe latency where interrupt delivery itself dominates
Fails badly on	nothing in particular	bursty or idle workloads (a spinning core doing nothing)	buffered I/O, sockets, anything without device polling — returns -EOPNOTSUPP

The three modes compared. What it shows: the two polling flags are not variants of one idea — SQPOLL moves submission off the application’s back by spending a kernel thread, while IOPOLL moves completion off the interrupt path by spending the caller’s own CPU inside the syscall. The insight: they attack opposite ends of the operation and can be combined, but each trades CPU for latency in a way that is only profitable at high sustained rates. “Just turn on the polling flags” is exactly the wrong instinct; the man page says as much about SQPOLL: “while this may sound immediately appealing as an automatic ‘go faster’ flag, evaluations should be done on a case-by-case basis” (io_uring_setup(2)).

IORING_SETUP_IOPOLL in Detail

IOPOLL is the mode most often misconfigured, because its requirements are strict and its failure is an errno rather than a slowdown. The check is in io_uring/rw.c at v6.12, in io_prep_rw:

if (ctx->flags & IORING_SETUP_IOPOLL) {
        if (!(kiocb->ki_flags & IOCB_DIRECT) || !file->f_op->iopoll)
                return -EOPNOTSUPP;          /* not O_DIRECT, or driver can't poll */
        kiocb->private = NULL;
        kiocb->ki_flags |= IOCB_HIPRI;       /* ask the block layer for polled completion */
        kiocb->ki_complete = io_complete_rw_iopoll;
        req->iopoll_completed = 0;
} else {
        if (kiocb->ki_flags & IOCB_HIPRI)
                return -EINVAL;              /* HIPRI without an IOPOLL ring is nonsense */
        kiocb->ki_complete = io_complete_rw;
}

Two conditions, both mandatory: the file must have been opened O_DIRECT (so IOCB_DIRECT is set) and its file_operations must supply an ->iopoll method, which in practice means a block device on a driver that implements queue polling. Fail either and every read on that ring returns -EOPNOTSUPP. Completions are then harvested by io_iopoll_check, which the io_uring_enter path enters instead of io_cqring_wait — it walks ctx->iopoll_list and asks the driver directly whether each request has landed. There is no interrupt and no softirq, which is the whole point: on a device with a 5 µs service time, the interrupt delivery and softirq dispatch can be a meaningful fraction of the latency, and polling removes it. Axboe’s document is careful to note that the crossover is hardware-dependent: “the numbers for when polling makes sense, either in terms of latency or overall IOPS rates, vary depending on the [device].”

stateDiagram-v2
    direction LR
    [*] --> Plain
    state "Plain ring<br/>submit: 1 syscall<br/>complete — interrupt" as Plain
    state "SQPOLL ring<br/>submit: 0 syscalls<br/>complete — interrupt" as SQP
    state "IOPOLL ring<br/>submit: 1 syscall<br/>complete — spin, no interrupt" as IOP
    state "SQPOLL + IOPOLL<br/>submit: 0 syscalls<br/>complete — poller thread spins" as BOTH

    Plain --> SQP: + IORING_SETUP_SQPOLL<br/>costs ~1 kernel thread
    Plain --> IOP: + IORING_SETUP_IOPOLL<br/>requires O_DIRECT + driver .iopoll
    SQP --> BOTH: + IORING_SETUP_IOPOLL
    IOP --> BOTH: + IORING_SETUP_SQPOLL
    BOTH --> [*]: lowest latency,<br/>highest CPU cost

The mode lattice. What it shows: the two flags are independent axes, so there are four reachable configurations, not three points on a line; each transition names both what it buys and what it costs. The insight: the bottom-right corner — SQPOLL plus IOPOLL — is the configuration behind the headline 1.7M-IOPS figure, and it consumes an entire dedicated core doing nothing but spinning on submission and completion queues. It is the right answer for a storage appliance saturating an NVMe device and the wrong answer for essentially everything else. In the v6.12 SQPOLL thread loop, this combination is visible directly: if ((ctx->flags & IORING_SETUP_IOPOLL) && !wq_list_empty(&ctx->iopoll_list)) { needs_sched = false; break; } — the poller refuses to sleep while polled I/O is still outstanding (io_uring/sqpoll.c).

---

SQPOLL — Zero Syscalls in Steady State

IORING_SETUP_SQPOLL is the most aggressive form of batching: it removes the syscall entirely from the steady-state submission path. The kernel spawns a dedicated thread that, per io_uring(7), “dequeues SQEs off the SQ as you add them and dispatches them for asynchronous processing.” The application fills SQEs and advances the SQ tail in shared memory; the poller thread notices the new tail and consumes the work — no io_uring_enter, no trap.

There is one subtlety that makes this safe rather than a busy-wait forever: the poller thread sleeps after sq_thread_idle milliseconds of inactivity to avoid burning a core when there is no work. When it sleeps, it sets the IORING_SQ_NEED_WAKEUP flag (1U << 0) in the SQ ring’s flags field. The application’s submission code must therefore check that flag after advancing the tail, and only if it is set issue one io_uring_enter with IORING_ENTER_SQ_WAKEUP (1U << 1) to wake the thread. So in the hot path — while traffic is steady and the thread is awake — submission costs zero syscalls; the rare wakeup call is paid only after an idle gap. This is verified against the v6.12 uapi header, where IORING_SQ_NEED_WAKEUP, IORING_ENTER_SQ_WAKEUP, and IORING_ENTER_SQ_WAIT (1U << 2, used to block until an SQ slot frees) are all defined.

/* steady-state SQPOLL submit loop, in spirit */
sqe = get_sqe(ring);            /* shared memory, no syscall   */
fill_sqe(sqe, ...);             /* opcode, fd, addr, len, ...  */
io_uring_smp_store_release(sq.tail, ++tail);   /* publish, barrier */
 
if (READ_ONCE(*sq.flags) & IORING_SQ_NEED_WAKEUP)   /* poller asleep? */
    io_uring_enter(fd, to_submit, 0,
                   IORING_ENTER_SQ_WAKEUP, NULL);   /* the only syscall */
/* else: nothing — the kernel poller already saw the new tail */

sequenceDiagram
    autonumber
    participant App as Application thread
    participant SQ as SQ ring (shared memory)
    participant Poll as SQPOLL kernel thread
    participant BE as Backend
    participant CQ as CQ ring (shared memory)

    Note over Poll: awake, spinning in io_sq_thread()
    App->>SQ: fill the SQE, then set array slot tail mod 8 to its index
    App->>SQ: store_release(sq.tail, tail+1)
    App->>SQ: read sq.flags — NEED_WAKEUP clear
    Note right of App: NO SYSCALL. The application<br/>has now submitted work.
    Poll->>SQ: io_sqring_entries(ctx) sees tail moved
    Poll->>BE: __io_sq_thread → io_submit_sqes
    BE-->>CQ: post CQE, publish cq.tail
    App->>CQ: poll cq.tail, read result
    Note right of App: NO SYSCALL. Round trip complete<br/>with ZERO boundary crossings.

    Note over Poll: idle for sq_thread_idle jiffies
    Poll->>SQ: atomic_or(IORING_SQ_NEED_WAKEUP, sq_flags)
    Poll->>Poll: smp_mb__after_atomic()
    Poll->>SQ: re-check io_sqring_entries() — still empty
    Poll->>Poll: schedule() — sleeps, core released
    App->>SQ: fill sqe — store_release(sq.tail, tail+1)
    App->>SQ: read sq.flags — NEED_WAKEUP SET
    App->>Poll: io_uring_enter(fd, n, 0, IORING_ENTER_SQ_WAKEUP, NULL)
    Note right of App: the ONLY syscall on this path,<br/>paid once per idle gap
    Poll->>SQ: atomic_andnot(NEED_WAKEUP, sq_flags) — resume

A full SQPOLL round trip, then the sleep/wake handoff. What it shows: in the steady state (steps 1–9) an operation is submitted and its result collected with no system call whatsoever — every arrow is a load or store against mapped memory. Steps 10–17 show the only path that costs a syscall: after sq_thread_idle of inactivity the poller publishes IORING_SQ_NEED_WAKEUP and sleeps, and the next submitter must notice that flag and ring the doorbell. The insight: the syscall is amortized not over a batch but over an idle gap, so a ring that is continuously busy pays nothing at all. This is the only Linux interface on which a full I/O round trip crosses the privilege boundary zero times.

The sleep handoff hides a genuine race, and the v6.12 source solves it with a barrier rather than a lock. If the poller set NEED_WAKEUP and went to sleep between an application’s tail store and its flag read, the work would sit unsubmitted forever. The io_uring/sqpoll.c code is explicit:

atomic_or(IORING_SQ_NEED_WAKEUP, &ctx->rings->sq_flags);
...
/*
 * Ensure the store of the wakeup flag is not
 * reordered with the load of the SQ tail
 */
smp_mb__after_atomic();
 
if (io_sqring_entries(ctx)) {   /* work appeared after we set the flag? */
        needs_sched = false;     /* then do NOT sleep */
        break;
}

Read together with the application’s side — publish tail, then read flags — this is the classic Dekker-style mutual-exclusion pattern: each party writes its own variable and then reads the other’s, with a full barrier in between, so at least one of them must observe the other’s write. Either the poller sees the new tail and stays awake, or the application sees NEED_WAKEUP and issues the wakeup call. It is impossible for both to miss.

stateDiagram-v2
    [*] --> Spinning
    Spinning: SPINNING<br/>NEED_WAKEUP clear<br/>app submits with 0 syscalls<br/>~1 core consumed
    Idling: IDLING<br/>no work seen, but<br/>jiffies < timeout<br/>still spinning
    Sleeping: SLEEPING<br/>NEED_WAKEUP SET<br/>0 CPU consumed<br/>app MUST ring the doorbell
    Spinning --> Idling: io_sqring_entries() == 0
    Idling --> Spinning: new SQ tail observed
    Idling --> Sleeping: idle > sq_thread_idle<br/>(default HZ = 1 second)<br/>set flag, barrier, re-check, schedule()
    Sleeping --> Spinning: io_uring_enter(IORING_ENTER_SQ_WAKEUP)<br/>wake_up on sq_data.wait<br/>clears NEED_WAKEUP
    Sleeping --> [*]: ring closed

The SQPOLL thread as a state machine. What it shows: three states with one flag — IORING_SQ_NEED_WAKEUP — visible to userspace, set on exactly one transition and cleared on exactly one. The insight to take: the flag is the application’s only signal about which submission protocol applies, and the check is not optional. The default sq_thread_idle is one second: ctx->sq_thread_idle = msecs_to_jiffies(p->sq_thread_idle); if (!ctx->sq_thread_idle) ctx->sq_thread_idle = HZ; — so a ring left with the default burns a core for a full second after its last operation. Setting it to a few milliseconds makes the thread far cheaper on bursty traffic at the price of one wakeup syscall per burst, which is usually the right trade.

One more SQPOLL subtlety worth knowing, because it silently caps throughput: a single poller thread (struct io_sq_data) can service several rings, and when it does, it round-robins with a hard fairness cap. In __io_sq_thread, if (cap_entries && to_submit > IORING_SQPOLL_CAP_ENTRIES_VALUE) to_submit = IORING_SQPOLL_CAP_ENTRIES_VALUE; where IORING_SQPOLL_CAP_ENTRIES_VALUE is 8 — but cap_entries is only true when !list_is_singular(&sqd->ctx_list), i.e. when more than one ring shares the thread (as happens with IORING_SETUP_ATTACH_WQ). A ring with its own dedicated poller submits its whole queue each pass; a ring sharing one takes at most 8 entries per visit.

The Privilege History — A Stale Fact Worth Correcting

Axboe’s design document states that “setting up an io_uring instance with IORING_SETUP_SQPOLL is a privileged operation. If the user doesn’t have sufficient privileges” the call fails with -EPERM. That is no longer true, and has not been since Linux 5.13. The progression is verifiable directly from the source at each tag:

Kernel	Check in io_sq_offload_create	Effect
5.10 and earlier	if (!capable(CAP_SYS_ADMIN)) goto err;	root, effectively
5.11–5.12	if (!capable(CAP_SYS_ADMIN) && !capable(CAP_SYS_NICE)) goto err;	CAP_SYS_NICE suffices
5.13 onward, incl. 6.12	no capability check at all	unprivileged SQPOLL

The SQPOLL privilege requirement over time, verified by fetching fs/io_uring.c (later io_uring/sqpoll.c) at each tag from raw.githubusercontent.com and grepping for the capability test. The v6.12 io_uring/sqpoll.c contains no capable() call whatsoever. The man page corroborates: “5.11 also allows using this as non-root, if the user has the CAP_SYS_NICE capability. In 5.13 this requirement was also relaxed, and no special privileges are needed for SQPOLL in newer kernels” (io_uring_setup(2)). The insight: this is a good illustration of why the design document must be read as a 2019 snapshot rather than a specification — three separate claims in it (the 4096 entry ceiling, the SQPOLL privilege requirement, and the requirement to pre-register files before using SQPOLL, lifted in 5.11) are now wrong.

The trade-off is real and worth stating bluntly: SQPOLL dedicates a kernel thread to spinning , so it costs a CPU (or a fraction of one if sq_thread_idle is tuned aggressively). It is a throughput-for-CPU trade, sensible for a dedicated high-IOPS storage or network server and wasteful for a sporadically-active process. It also changes buffer-lifetime rules: without SQPOLL a non-fixed buffer pointer need only stay valid until io_uring_enter returns, but with SQPOLL — since submission is asynchronous — the pointer must stay valid until completion (io_uring(7)).

---

Ordering Inside a Batch — Linked SQEs

Batching creates a problem it must then solve. Once you submit N operations in one call, the kernel is free to dispatch them concurrently and complete them out of order — which is exactly the point, but it breaks any workflow where step k+1 depends on step k. Writing a file and then fsync-ing it is the obvious case: submitted as an unordered batch, the fsync may run first and sync nothing. The naive fix is to split the batch — submit the write, wait for its CQE, then submit the fsync — which reintroduces exactly the per-operation syscall this whole design exists to eliminate.

IOSQE_IO_LINK is the answer. Setting it in sqe->flags means “the next SQE in this submission is dependent on me,” and a run of consecutive SQEs each carrying the flag forms a chain, terminated by the first SQE that does not carry it. The v6.12 uapi header comments are terse — /* links next sqe */ for IOSQE_IO_LINK and /* like LINK, but stronger */ for IOSQE_IO_HARDLINK — so the semantics have to be read out of io_submit_sqe in io_uring/io_uring.c:

if (unlikely(link->head)) {                     /* we are mid-chain */
        link->last->link = req;                 /* append to the chain */
        link->last = req;
        if (req->flags & IO_REQ_LINK_FLAGS)     /* LINK or HARDLINK: more to come */
                return 0;                       /* do NOT issue yet */
        /* last request of the link, flush it */
        req = link->head;                       /* issue the HEAD; the rest follow it */
        link->head = NULL;
        ...
} else if (unlikely(req->flags & (IO_REQ_LINK_FLAGS | ...))) {
        if (req->flags & IO_REQ_LINK_FLAGS) {
                link->head = req;               /* start a new chain */
                link->last = req;
        }
        ...
        return 0;
}
io_queue_sqe(req);                              /* unlinked request: issue immediately */

The critical detail is that a linked SQE is not issued when it is submitted. It is parked on link->head, and only when the terminating (unflagged) SQE arrives is the chain’s head issued; each subsequent member starts when its predecessor completes. So a five-deep chain is still submitted in one io_uring_enter — the batching win is preserved — but executes strictly serially inside the kernel.

The difference between the two link flags is failure handling. Under IOSQE_IO_LINK, if a member fails (returns a negative res), the rest of the chain is cancelled, each posting a CQE with res = -ECANCELED. Under IOSQE_IO_HARDLINK, ordering is still enforced but a failure does not sever the chain — the successor runs anyway. The v6.12 source shows the cancellation propagating through req_fail_link_node(head, -ECANCELED) in io_submit_fail_init, whose comment also flags a subtlety: “Avoid breaking links in the middle as it renders links with SQPOLL unusable. Instead of failing eagerly, continue assembling the link if applicable and mark the head with REQ_F_FAIL.”

flowchart TD
    subgraph BATCH["ONE io_uring_enter(to_submit=6)"]
        direction TB
        S1["sqe[0] OPENAT<br/>flags: IOSQE_IO_LINK"]
        S2["sqe[1] WRITE<br/>flags: IOSQE_IO_LINK"]
        S3["sqe[2] FSYNC<br/>flags: IOSQE_IO_LINK"]
        S4["sqe[3] CLOSE<br/>flags: 0 ← terminates chain"]
        S5["sqe[4] RECV fd=9<br/>flags: 0 (independent)"]
        S6["sqe[5] RECV fd=11<br/>flags: 0 (independent)"]
    end
    S1 -->|"must complete first"| S2
    S2 -->|"must complete first"| S3
    S3 -->|"must complete first"| S4
    S1 -.->|"if res &lt; 0 with IO_LINK:<br/>rest posted as -ECANCELED"| CANCEL["sqe[1..3] → res = -ECANCELED"]
    S5 --> PAR["issued immediately,<br/>concurrently with the chain"]
    S6 --> PAR
    S4 --> DONE["4 CQEs, in chain order"]
    PAR --> DONE2["2 CQEs, whenever the<br/>sockets deliver"]

A six-SQE batch containing one four-deep link chain and two independent operations. What it shows: IOSQE_IO_LINK imposes a serial dependency within the batch without splitting the batch — all six SQEs still go in one io_uring_enter — while unlinked SQEs in the same submission run concurrently alongside the chain. The insight: links let you express a whole open-write-fsync-close workflow as a single boundary crossing, which is the difference between io_uring being a faster syscall and io_uring being a programmable one. The dashed edge is the failure semantics: with IOSQE_IO_LINK a failure anywhere collapses the remainder into -ECANCELED CQEs (so you still get one CQE per SQE and your accounting stays balanced); with IOSQE_IO_HARDLINK the successors run regardless.

Two neighbouring flags round out the intra-batch controls and are easy to confuse with links:

• IOSQE_IO_DRAIN (/* issue after inflight IO */) is a barrier, not a link: the flagged SQE is held until every previously submitted request on the ring has completed, then runs. It orders one request against the whole ring rather than against its immediate predecessor, and it is correspondingly expensive — it drains your pipeline.
• IOSQE_CQE_SKIP_SUCCESS (/* don't post CQE if request succeeded */) suppresses the completion entry when res >= 0. On a long chain where only the final result matters, this removes N−1 CQEs from the completion ring, which both saves CQ space and saves the application from walking entries it would discard. It pairs naturally with links.

---

Security Restrictions — State as of 6.12 / 6.18 LTS

An honest note cannot present io_uring as a pure win, because several large operators have concluded that it is not safe to expose to untrusted code and have turned it off. The reason is structural rather than incidental: io_uring gives userspace a shared-memory channel into a kernel component that executes arbitrary I/O opcodes on the caller’s behalf, sometimes from a different task (the SQPOLL thread or an io-wq worker) with borrowed credentials, against pre-pinned memory and pre-resolved file references. That is an unusually rich set of exploitation primitives — asynchronous lifetime, cross-task execution, and long-lived kernel-held references to user resources — and it has produced a corresponding run of high-severity bugs.

The most-cited data point comes from Google’s own security blog, and it is worth quoting exactly rather than paraphrasing, because the widely-repeated version of it drifts. In Learnings from kCTF VRP’s 42 Linux kernel exploits submissions (2023-06-14), Tamás Koczka writes: “in the past year, there has been a clear trend: 60% of the submissions exploited the io_uring component of the Linux kernel (we paid out around 1 million USD for io_uring alone). Furthermore, io_uring vulnerabilities were used in all the submissions which bypassed our mitigations” (Google Online Security Blog). The window is “the past year” relative to June 2023 — roughly mid-2022 to mid-2023 — not the calendar year 2022, and the denominator is the 42 kCTF exploit submissions, not Linux kernel exploits at large. The same post states Google’s conclusion without hedging: “we currently consider it safe only for use by trusted components.”

The measures Google describes, again verbatim from that post:

Environment	Action taken (as stated in June 2023)
ChromeOS	“We disabled io_uring (while we explore new ways to sandbox it).”
Android	“Our seccomp-bpf filter ensures that io_uring is unreachable to apps. Future Android releases will use SELinux to limit io_uring access to a select few system processes.”
GKE Autopilot	“We are investigating disabling io_uring by default.” — note this is not a statement that it was disabled
Google production servers	“It is disabled on production Google servers.”

Google’s stated io_uring restrictions. What it shows: four different environments, three different mechanisms (kernel build config, seccomp-bpf, SELinux), and one that was still under consideration at the time of writing. The insight: the correction that matters is the GKE Autopilot row — secondary write-ups routinely report it as “Google disabled io_uring in GKE,” which the primary source does not say. This is exactly the kind of claim that decays: the post is from 2023-06-14 and Google’s posture may well have hardened since, so treat the table as a dated snapshot, not current state.

Beyond Google, the container ecosystem made the same call by omission. Docker/moby’s default seccomp profile is an allowlist — "defaultAction": "SCMP_ACT_ERRNO", "defaultErrnoRet": 1 — and io_uring_setup, io_uring_enter, and io_uring_register do not appear in it (verified by grep against moby/profiles seccomp/default.json on main, 2026-09-04: zero occurrences of any of the three names). The legacy io_setup/io_submit aio calls are listed, as are epoll_create1 and epoll_wait. The practical consequence: a container running under Docker’s default profile gets EPERM from io_uring_setup and cannot use io_uring at all — a fact that surprises people who benchmark on the host and deploy in a container. Running with --security-opt seccomp=unconfined, or a custom profile that adds the three syscalls, is what restores it.

The specific bugs behind the policy are worth naming rather than gesturing at, since they cluster on precisely the features this note has been praising:

CVE	Published	Component	Nature
CVE-2021-41073	2021-09-19	loop_rw_iter, 5.10–5.14.6	IORING_OP_PROVIDE_BUFFERS frees a kernel buffer → local privilege escalation
CVE-2022-29582	2022-04-22	io_uring timeouts, before 5.17.3	Use-after-free race; exploitable by a local user with no user-namespace access
CVE-2023-2598	2023-06-01	io_sqe_buffer_register, io_uring/rsrc.c	Out-of-bounds access to physical memory past the end of a registered buffer → “full local privilege escalation”
CVE-2023-21400	2023-07-13	multiple functions in io_uring.c	Kernel memory corruption from improper locking → local privilege escalation
CVE-2024-0582	2024-01-16	IORING_REGISTER_PBUF_RING	Memory leak on register → mmap → free; crash or privilege escalation

Five representative io_uring CVEs, each verified against its NVD advisory record rather than a secondary retelling. What it shows: the fixing sites are rsrc.c (buffer registration), the provided-buffer ring, the timeout machinery, and locking in the core submission path. The insight to take: these are not random memory-safety bugs scattered through a large subsystem — they concentrate in exactly the mechanisms that make io_uring fast. Registration means the kernel holds long-lived references to user pages, so a lifetime mistake there is an OOB write to physical memory (CVE-2023-2598). Asynchronous, cross-task execution means locking is genuinely hard (CVE-2023-21400) and lifetimes are genuinely racy (CVE-2022-29582). The performance design and the attack surface are the same design.

The Kernel’s Own Knob — kernel.io_uring_disabled

Everything above is deployment policy layered around an unchanged kernel. The kernel’s own system-wide control knob, the kernel.io_uring_disabled sysctl, arrived in Linux 6.6 — verified by existence-checking io_uring/io_uring.c at consecutive tags on raw.githubusercontent.com: zero occurrences of io_uring_disabled at v6.4 and v6.5, seven at v6.6. (This corrects an earlier claim in this note that it came from “5.6-era hardening work”; that was wrong by a full year of releases. The LWN write-up that discussed the patch, Add a sysctl to disable io_uring system-wide, is dated 2023, consistent with a 6.6 merge.) Its definition is identical in the v6.12 and v6.18 LTS sysctl/kernel.rst (verified by curl of both pinned tags):

• 0 — all processes may create io_uring instances. This is the kernel’s default.
• 1 — io_uring_setup() fails with -EPERM for unprivileged processes not in the io_uring_group group; privileged (CAP_SYS_ADMIN) processes and group members still may. The companion kernel.io_uring_group sysctl names that GID; when it is -1 (the default) only CAP_SYS_ADMIN processes qualify.
• 2 — io_uring_setup() always fails with -EPERM for everyone. Existing instances keep working.

The merged commit settles the provenance exactly. It is 76d3ccecfa186af3120e206d62f03db1a94a535f, “io_uring: add a sysctl to disable io_uring system-wide”, authored by Matteo Rizzo of Google on 2023-08-21, amended by Jeff Moyer (Red Hat) to add the io_uring_group knob and applied by Jens Axboe. Its message defines the three values verbatim: “When 0 (the default), all processes are allowed to create io_uring instances, which is the current behavior. When 1, io_uring creation is disabled (io_uring_setup() will fail with -EPERM) for unprivileged processes not in the kernel.io_uring_group group. When 2, calls to io_uring_setup() fail with -EPERM regardless of privilege.” The same commit adds the documentation stanza, which states the motivation in one sentence — “Prevents all processes from creating new io_uring instances. Enabling this shrinks the kernel’s attack surface.” The earlier v3 posting of the same series, from 2023-06-30, spells out the threat model: “Over the last few years we’ve seen many critical vulnerabilities in io_uring which could be exploited by an unprivileged process to gain control over the kernel… The goal of this patch is to give distros, system admins, and cloud providers a way to reduce the risk of privilege escalation through io_uring where disabling it with seccomp or at compile time is not practical” (LWN’s archive of the posting). An August-2023 commit lands in the 6.6 merge window, corroborating the tag-existence result above exactly.

Source-routing note — the Anubis trigger is the User-Agent, not the host

Measured 2026-09-04 against the same git.kernel.org cgit URL: sending a browser User-Agent (-A "Mozilla/5.0 ...") returns HTTP 200 with a 7.7 KB page titled “Making sure you’re not a bot!” — the Anubis proof-of-work challenge — while plain curl with no -A flag returns the real cgit HTML at HTTP 200. The bot check is triggered by the spoofed User-Agent, so the usual workaround of adding a browser UA is exactly what causes the block here. A status-code check catches neither case; verify the body.

This commit was consequently confirmed by two independent routes that agree: GitHub’s mirror at https://github.com/torvalds/linux/commit/<full-40-char-sha>.patch (SHA obtained from https://github.com/torvalds/linux/commits/v6.6/io_uring/io_uring.c.atom), and git.kernel.org cgit at .../log/?qt=grep&q=io_uring_disabled plus .../patch/?id=<full 40-char sha> fetched with no UA override. Note that a truncated SHA silently returns an empty cgit page, so always use the full 40 characters.

The enforcement is a single function called at the very top of the syscall, in io_uring/io_uring.c at v6.12:

static inline bool io_uring_allowed(void)
{
        int disabled = READ_ONCE(sysctl_io_uring_disabled);
        kgid_t io_uring_group;
 
        if (disabled == 2)
                return false;                      /* 2: nobody, not even root */
 
        if (disabled == 0 || capable(CAP_SYS_ADMIN))
                return true;                       /* 0: everybody; else CAP_SYS_ADMIN */
 
        io_uring_group = make_kgid(&init_user_ns, sysctl_io_uring_group);
        if (!gid_valid(io_uring_group))
                return false;                      /* group == -1 → no group qualifies */
 
        return in_group_p(io_uring_group);         /* 1: members of io_uring_group */
}
 
SYSCALL_DEFINE2(io_uring_setup, u32, entries, struct io_uring_params __user *, params)
{
        if (!io_uring_allowed())
                return -EPERM;
        return io_uring_setup(entries, params);
}

Two things are easy to get wrong and are settled by reading this. First, at value 2 even CAP_SYS_ADMIN is refused — the disabled == 2 test returns before the capability check, so this is a genuine system-wide off switch and not merely a privilege gate. Second, at value 1 the fallback when kernel.io_uring_group is left at its default of -1 is return false, so value 1 with no group configured behaves as “CAP_SYS_ADMIN only.”

flowchart TD
    CALL["io_uring_setup(entries, params)"] --> D{"sysctl_io_uring_disabled"}
    D -->|"== 2"| DENY["return -EPERM<br/>NO exception for root"]:::deny
    D -->|"== 0 (upstream default)"| ALLOW["create the ring"]:::allow
    D -->|"== 1"| CAP{"capable(CAP_SYS_ADMIN)?"}
    CAP -->|yes| ALLOW
    CAP -->|no| GID{"kernel.io_uring_group<br/>a valid GID?"}
    GID -->|"no (default -1)"| DENY
    GID -->|yes| MEM{"in_group_p(group)?"}
    MEM -->|yes| ALLOW
    MEM -->|no| DENY
    SECCOMP["Docker default seccomp profile:<br/>io_uring_setup not on the allowlist"]:::deny -.->|"blocks BEFORE the kernel<br/>function is even reached"| CALL
    classDef deny fill:#fdd,stroke:#c33
    classDef allow fill:#dfd,stroke:#3a3

The full gate on io_uring_setup, transcribed from io_uring_allowed() at v6.12 plus the container layer above it. What it shows: three sysctl values producing four distinct outcomes, and a second, independent gate — seccomp — that fires earlier and is invisible to the sysctl entirely. The insight for debugging: an unexpected EPERM from io_uring_setup has at least two possible causes that need different fixes. Read /proc/sys/kernel/io_uring_disabled first; if it is 0 and you are still getting EPERM, you are almost certainly inside a container whose seccomp profile never allowed the syscall, and no amount of sysctl tuning will help.

Uncertain

A common conflation worth flagging: several secondary write-ups state the default io_uring_disabled value is 2. That is not the upstream kernel default — the v6.12 and v6.18 LTS docs both say value 0 (“This is the default setting”) and the kernel ships with io_uring enabled. The value-2 claim conflates the mainline kernel default with the hardened default chosen by specific distributions or vendors (e.g. certain enterprise/Google images set it to 1 or 2 out of the box). Verify the effective default per distribution and kernel build, not from the upstream source alone. Reason: source default (0) vs vendor policy (often 1/2) diverge. To resolve: read /proc/sys/kernel/io_uring_disabled on the actual target system. uncertain

---

Failure Modes and Misunderstandings

• “io_uring is always faster than blocking syscalls.” Not for low-concurrency, low-IOPS workloads. The ring setup, the memory ordering discipline, and (for SQPOLL) the dedicated thread are pure overhead if you only do a handful of operations. The win materializes when there are many in-flight operations to amortize across — that is exactly the regime where one-syscall-per-op hurts.
• SQPOLL on the wrong workload. A poller thread spinning on an idle ring wastes a core. Tune sq_thread_idle, or do not use SQPOLL for bursty workloads.
• Submitting more than the SQ depth in flight. The SQE lifetime ends once the kernel consumes it, so an application can drive more pending requests than the SQ ring size. But if it drives more than the CQ ring can hold, it risks CQ overflow — surfaced via the IORING_SQ_CQ_OVERFLOW flag (1U << 1, v6.12) and, on modern kernels with IORING_FEAT_NODROP, stashed internally rather than dropped. By default the CQ ring is twice the SQ size to give headroom.
• io_uring_setup returns -EPERM unexpectedly. Almost always the io_uring_disabled sysctl (value 1 or 2) or a seccomp profile (Docker default) blocking the syscall — not a code bug.
• Forgetting the SQPOLL wakeup check. If the application never inspects IORING_SQ_NEED_WAKEUP, submissions silently stall once the poller sleeps, because nothing tells the kernel to wake up. The symptom is nasty: everything works perfectly under load and hangs the moment traffic pauses for longer than sq_thread_idle, so it survives benchmarking and fails in production at 3 a.m.
• IOPOLL on the wrong file. IORING_SETUP_IOPOLL is a ring-wide flag, but the check is per operation: io_prep_rw returns -EOPNOTSUPP for any read or write whose file was not opened O_DIRECT or whose driver lacks an ->iopoll method. So an IOPOLL ring that also handles a socket or a buffered file does not degrade gracefully — those operations simply fail. Use a separate ring for polled storage.
• Assuming CQE order matches SQE order. Completions arrive in whatever order the backends finish. Code that walks the CQ ring and assumes cqe[k] corresponds to sqe[k] will silently mis-attribute results as soon as one operation is slower than another. user_data is the only correspondence, which is precisely why it exists — use it, and prefer storing a pointer to your own request object rather than an index that can go stale.
• Buffer lifetime under SQPOLL. Without SQPOLL, a non-registered buffer pointer need only stay valid until io_uring_enter returns. With SQPOLL, submission happens asynchronously in another thread, so the pointer must stay valid until completion (io_uring(7)). Turning on SQPOLL as a “go faster” flag on code written for the plain mode is a straightforward use-after-free.
• Expecting linked SQEs to be issued at submission time. They are not: a chain’s head is issued only when the terminating unflagged SQE arrives. A batch that ends with a linked SQE and no terminator leaves the whole chain parked — nothing runs, and no CQE appears, until a subsequent submission terminates it.
• Benchmarking on the host, deploying in a container. Docker’s default seccomp profile does not allowlist io_uring_setup, so the containerized build gets EPERM and (if the code has a fallback) silently runs the slow path. This is the single most common way a measured io_uring speedup fails to materialize in production.

---

Alternatives and When to Choose Them

The honest way to place io_uring is on two axes at once: how many syscalls does N operations cost, and what does the API actually let you express. Laid out that way the family relationships are clear.

Mechanism	Model	Syscalls for N ops on M fds	Works on regular files?	API shape	Chief limitation
Blocking read/write	synchronous	N	yes (but blocks)	one call, one op	one blocked thread per outstanding op
[[Vectored IO readv and writev|readv/writev]]	synchronous, batched buffers	1 per fd	yes	one call, many segments, one fd, one op type	no concurrency, no heterogeneity
[[sendmmsg and recvmmsg Batched Socket Calls|sendmmsg/recvmmsg]]	synchronous, batched messages	1 per socket	no (sockets only)	one call, many datagrams, one socket	one socket, one direction
[[The poll and select Syscalls|select/poll]]	readiness	1 wait + N I/O = N+1, and the wait is O(total watched)	no — always “ready”, uselessly	rebuild the whole fd set every call	O(n) rescan per call; fd set copied in and out
[[epoll and Scalable Readiness Notification|epoll]]	readiness	1 wait + N I/O = N+1, wait is O(ready)	no — epoll_ctl(ADD) returns EPERM	register once, drain a ready list	still one syscall per ready operation; blind to file I/O
Linux native aio	completion	2 (submit + io_getevents)	O_DIRECT only	submit array of iocb	buffered I/O silently synchronous; 104 bytes copied per op
io_uring	completion	1, or 0 with SQPOLL	yes, buffered or direct	shared rings, heterogeneous opcodes, linkable	complexity; memory-ordering discipline; restricted or disabled in many environments

The batching and multiplexing family on one grid. What it shows: the three columns that actually differentiate them are the syscall count, the regular-file column, and the API-shape column. readv and sendmmsg batch along a narrow axis (one fd, one operation type); readiness interfaces batch only the notification; io_uring is the only row that batches heterogeneous operations across arbitrary descriptors and covers regular files. The insight: the “works on regular files” column is the one that most often decides the choice in practice. If your server does only socket I/O, epoll is simpler, universally available, unrestricted by container policy, and costs you one extra syscall per ready connection — frequently the right answer. If you also read files, epoll gives you nothing for that half of the workload and you end up with a thread pool bolted alongside it; io_uring covers both in one mechanism. See epoll and Scalable Readiness Notification for the readiness column in depth.

• Plain blocking syscalls (read/write on a thread pool): simplest, fine for modest concurrency; the Go runtime hides exactly this behind goroutines, parking a goroutine across a blocking syscall without dedicating an OS thread.
• epoll + non-blocking syscalls: the classic readiness-based reactor. Still one syscall per ready operation after the epoll_wait; io_uring is completion-based and batches the operations themselves, not just the readiness notification.
• Vectored IO readv and writev and sendmmsg and recvmmsg Batched Socket Calls: lightweight batching for a single fd / single operation type, no ring setup. Reach for these when the batching need is narrow.
• io_uring: when you have high concurrency, heterogeneous operations, and the per-syscall overhead is your measured bottleneck — and the security policy of your environment permits it.

---

See Also

• epoll and Scalable Readiness Notification — the readiness-based counterpart to this note’s completion-based model; read the two together, since the readiness/completion distinction is the axis that explains both
• The io_uring Submission and Completion Queues — the SQ/CQ ring mechanics, SQEs/CQEs, indices, and barriers this note rests on
• Why System Calls Are Expensive — the fixed per-trap cost that io_uring amortizes away
• sendmmsg and recvmmsg Batched Socket Calls — narrow, single-operation batching, for contrast
• Vectored IO readv and writev — scatter/gather batching for one fd
• The net Package and the Poller — how the Go runtime hides blocking syscalls behind a poller, a different point in the design space
• Linux System Call Interface MOC — parent map (§9, Batching Syscalls)
• Linux Block Layer and Storage MOC / Linux Networking Stack MOC — where io_uring’s storage and network fast paths actually live