vault

AF_XDP Zero-Copy Sockets

19 min read

AF_XDP Zero-Copy Sockets

AF_XDP is a socket address family “optimized for high performance packet processing” (af_xdp.rst) that lets a userspace application receive and transmit raw Ethernet frames with minimal — and, on capable drivers, zero — copying. An AF_XDP socket (universally abbreviated XSK) is created with the ordinary socket(AF_XDP, SOCK_RAW, 0) call, but its data plane bypasses the entire kernel network stack: an XDP program running in the driver does bpf_redirect_map() into an XSKMAP, depositing the frame directly into a region of memory the application registered with the kernel called the UMEM (user memory). The application and kernel pass ownership of UMEM frames back and forth through four lock-free single-producer/single-consumer rings — RX, TX, FILL, and COMPLETION — so packets move between NIC and userspace without read()/write() copies and without the per-packet sk_buff allocation that dominates the normal path. AF_XDP is the kernel’s answer to userspace fast-packet frameworks like DPDK, but it keeps the NIC a normal kernel netdev. This note is the networking datapath view; the XDP verdict machinery is in XDP Express Data Path and the eBPF map/verifier machinery in Linux eBPF MOC — cross-linked, not duplicated.

This note is written against Linux 6.12 LTS (released 2024-11-17), with the ring/UMEM ABI and xsk.c receive paths verified against the v6.12 source tree. AF_XDP has been in the kernel since 4.18 (2018); the struct xdp_umem_reg, struct xdp_desc, the four-ring model, the XDP_COPY/XDP_ZEROCOPY/XDP_USE_NEED_WAKEUP/XDP_SHARED_UMEM bind flags, and multi-buffer (XDP_USE_SG) support described here are all present and verified in v6.12.

Mental Model — A Conveyor Belt of Frames Shared With the Kernel

The hardest part of AF_XDP is the ownership dance, and the clearest mental model is a conveyor belt of fixed-size trays (frames) running between two workers (the kernel and your application). The belt of trays is the UMEM: one contiguous block of memory you allocate (with malloc, mmap, or huge pages) and register with the kernel, sliced into equal chunks. At any instant each tray is owned by exactly one side, and the four rings are the four hand-off points where a tray crosses from one owner to the other. You never copy a packet between kernel and userspace; you just pass the address of the tray it sits in.

flowchart LR
  subgraph UMEM["UMEM (one shared memory region, sliced into equal frames)"]
    F["frame 0 | frame 1 | frame 2 | ... | frame N"]
  end

  subgraph APP["Userspace application (owns the rings)"]
    direction TB
    PROC["process packet<br/>(your userspace stack)"]
  end

  subgraph KERN["Kernel + driver"]
    direction TB
    DRV["driver DMA + XDP redirect"]
  end

  APP -->|"FILL ring: 'these trays are empty, fill them'"| KERN
  KERN -->|"RX ring: 'this tray now holds a received packet'"| APP
  APP -->|"TX ring: 'send the packet in this tray'"| KERN
  KERN -->|"COMPLETION ring: 'this tray was sent, reuse it'"| APP

The four-ring ownership protocol of AF_XDP. What it shows: the FILL and RX rings form the receive loop (app hands the kernel empty frames on FILL, kernel hands back filled frames on RX); the TX and COMPLETION rings form the transmit loop (app submits frames to send on TX, kernel returns transmitted frames on COMPLETION). All four carry only frame addresses (offsets into the UMEM), never packet bytes. The insight to take: a packet never crosses the user/kernel boundary by being copied — only the ownership of a fixed memory tray crosses, by passing its address through a ring. FILL+RX are the RX path, TX+COMPLETION are the TX path; the FILL/COMPLETION pair belongs to the UMEM, the RX/TX pair belongs to each socket.

The UMEM — User-Registered Frame Memory

The UMEM is, per the docs, “a region of virtual contiguous memory, divided into equal-sized frames” (af_xdp.rst). The application allocates this memory itself — the kernel does not allocate it — then registers it with setsockopt(fd, SOL_XDP, XDP_UMEM_REG, ...) passing a struct xdp_umem_reg from include/uapi/linux/if_xdp.h:

struct xdp_umem_reg {
    __u64 addr;             /* start of the packet data area (your mmap'd region) */
    __u64 len;              /* total length of that area in bytes */
    __u32 chunk_size;       /* size of each frame; only 2K or 4K (v6.12) */
    __u32 headroom;         /* bytes reserved at the front of each frame */
    __u32 flags;            /* e.g. XDP_UMEM_UNALIGNED_CHUNK_FLAG */
    __u32 tx_metadata_len;  /* per-chunk TX metadata reservation */
};

Walking the fields: addr/len describe the whole region; chunk_size divides it into frames — “It can only be 2K or 4K at the moment” (af_xdp.rst) — so a 128 KiB region with 2 KiB chunks holds 64 frames and a maximum packet size of 2 KiB. headroom reserves N bytes at the front of each frame for the application to use (e.g. to prepend a header without shifting data). A descriptor in any ring references a frame by its addr, which is simply “an offset within the entire UMEM region” — so frame k lives at offset k * chunk_size.

There are two addressing modes. In aligned mode (the default), the kernel masks the low bits of the address to the chunk boundary — “for a chunk size of 2k, the log2(2048) LSB of the addr will be masked off, meaning that 2048, 2050 and 3000 refers to the same chunk.” In unaligned mode (XDP_UMEM_UNALIGNED_CHUNK_FLAG), the address is left untouched, letting a packet start at an arbitrary offset within the region — useful for placing a packet so its payload lands on a page boundary, at the cost of the application managing offsets itself. A UMEM is bound to one netdev and one queue id via the bind() call.

The Four Rings — RX, TX, FILL, COMPLETION

There are exactly four kinds of ring, “all rings are single-producer/single-consumer” (af_xdp.rst), which is why they need no locks but do need the application to serialize its own access if multiple threads touch one ring. Two rings belong to the socket (RX, TX) and two belong to the UMEM (FILL, COMPLETION). For a setup of four sockets all doing RX and TX sharing one UMEM, you have one FILL ring, one COMPLETION ring, four RX rings, four TX rings.

Each ring is a classic head/tail circular buffer: a producer writes at the index in producer and increments it; a consumer reads at consumer and increments it. Ring sizes must be a power of two. The rings are created with the XDP_{RX,TX,UMEM_FILL,UMEM_COMPLETION}_RING setsockopts and then mmap()’d into the application’s address space at fixed page offsets (XDP_PGOFF_RX_RING, XDP_PGOFF_TX_RING, XDP_UMEM_PGOFF_FILL_RING, XDP_UMEM_PGOFF_COMPLETION_RING), so the application reads and writes them as plain memory with no syscall per packet.

The four rings carry two descriptor types. The FILL and COMPLETION rings carry bare __u64 UMEM addresses. The RX and TX rings carry a struct xdp_desc (include/uapi/linux/if_xdp.h):

struct xdp_desc {
    __u64 addr;    /* offset into UMEM where this frame's data sits */
    __u32 len;     /* length of the packet data */
    __u32 options; /* e.g. XDP_PKT_CONTD for multi-buffer */
};

The four rings divide into two loops:

  • Receive loop (FILL → RX). The application produces empty UMEM addresses onto the FILL ring (“these trays are available; put received packets in them”). The kernel consumes a FILL address, the driver places (or DMAs) an incoming packet into that frame, and the kernel produces a descriptor onto the RX ring pointing at that now-full frame, with addr and len set. The application consumes from the RX ring to get its packets. Crucially: “If no frames have been passed to kernel via the FILL ring, no descriptors will (or can) appear on the RX ring” — forget to refill and receive stalls.

  • Transmit loop (TX → COMPLETION). The application fills a struct xdp_desc (addr, len, options) and produces it onto the TX ring, then triggers transmit (see need_wakeup below). When the NIC has finished sending, the kernel produces the frame’s address onto the COMPLETION ring, signalling “this tray was sent completely and can be reused, for either TX or RX.” The addresses appearing in COMPLETION are exactly those previously submitted on TX.

A subtle but important property follows: because RX and TX can share one UMEM, “a packet does not have to be copied between RX and TX” — you can receive a frame, rewrite it in place, and submit the same frame on TX, achieving zero-copy forwarding. And to hold a packet for possible retransmit, the descriptor can simply be re-pointed at a different frame and reused immediately.

Zero-Copy vs Copy Mode — Where the Data Actually Lives

The single most important performance distinction is zero-copy vs copy mode, and the kernel’s receive dispatch in net/xdp/xsk.c makes the difference concrete:

static int xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp)
{
    ...
    if (xdp->rxq->mem.type == MEM_TYPE_XSK_BUFF_POOL) {
        len = xdp->data_end - xdp->data;
        return xsk_rcv_zc(xs, xdp, len);   /* ZERO-COPY: driver DMA'd into UMEM already */
    }
 
    err = __xsk_rcv(xs, xdp, len);          /* COPY: memcpy into a freshly-allocated UMEM frame */
    if (!err)
        xdp_return_buff(xdp);
    return err;
}

In zero-copy mode (XDP_ZEROCOPY), the NIC driver has been told to DMA incoming frames directly into the UMEM frames — the memory pool type is MEM_TYPE_XSK_BUFF_POOL. When such a frame is redirected to the socket, xsk_rcv_zc() simply hands ownership of that already-populated frame to the RX ring. No byte is copied: the data the NIC wrote by DMA is the data the application reads. This requires the driver to implement the XSK pool callbacks (XDP_SETUP_XSK_POOL in the ndo_bpf command set) and DMA-map the UMEM; drivers like i40e, ice, ixgbe, mlx5, and stmmac support it.

In copy mode (XDP_COPY), the driver DMAs into its own receive buffers as usual, and on redirect the kernel’s __xsk_rcv() allocates a UMEM frame and memcpys the packet into it:

xsk_xdp = xsk_buff_alloc(xs->pool);
...
memcpy(xsk_xdp->data - meta_len, copy_from, rem);   /* the copy that "copy mode" is named for */

So copy mode still avoids the kernel stack and the sk_buff overhead, but pays one memcpy per packet from the driver buffer into the UMEM. It is the universal fallback that works on any driver. Per the docs: “When you bind to a socket, the kernel will first try to use zero-copy. If zero-copy is not supported, it will fall back on using copy mode” (af_xdp.rst). You can force either with the XDP_ZEROCOPY or XDP_COPY bind flag — forcing a mode the driver can’t provide makes bind() fail rather than silently degrade. After binding, getsockopt(XDP_OPTIONS) returns XDP_OPTIONS_ZEROCOPY to tell you which you got.

There is also a third mode at the XDP level — generic (SKB) mode — where the XDP program runs after the skb is built and AF_XDP copies out of the skb; this is the slowest fallback for drivers with no native XDP at all. (The native-vs-generic XDP distinction itself belongs to XDP Express Data Path.)

How XDP Redirects Into an XSK — The XSKMAP

AF_XDP cannot receive anything without an XDP program redirecting to it. The bridge is the XSKMAP (BPF_MAP_TYPE_XSKMAP): a BPF map whose values are XSK sockets. The userspace application inserts its socket file descriptor into the map at some index (via the bpf() syscall), and an XDP program redirects matching frames to that index with bpf_redirect_map(&xsks_map, index, ...). The kernel-side dispatch, from net/core/filter.c, routes XSKMAP redirects specially:

if (map_type == BPF_MAP_TYPE_XSKMAP)
    return __xdp_do_redirect_xsk(ri, dev, xdp, xdp_prog);

which calls __xsk_map_redirect() → xsk_rcv() (the zero-copy/copy branch above). The canonical redirect program (shipped in libxdp) keys on the receive queue index:

SEC("xdp_sock") int xdp_sock_prog(struct xdp_md *ctx)
{
    int index = ctx->rx_queue_index;
    /* A set entry means an AF_XDP socket is bound to this queue. */
    if (bpf_map_lookup_elem(&xsks_map, &index))
        return bpf_redirect_map(&xsks_map, index, 0);
    return XDP_PASS;   /* no socket for this queue → normal stack */
}

The redirect is validated: “XDP validates that the XSK in that map was indeed bound to that device and ring number. If not, the packet is dropped. If the map is empty at that index, the packet is also dropped” (af_xdp.rst). This validation appears as xsk_rcv_check() in net/xdp/xsk.c, which rejects the frame if xs->dev != xdp->rxq->dev || xs->queue_id != xdp->rxq->queue_index. The consequence is the rule from the docs: a loaded XDP program with at least one XSK in the XSKMAP is mandatory to get any traffic to userspace through an XSK. A socket bound to queue 17 of eth0 only ever sees traffic the NIC steered to queue 17 — which is why AF_XDP setups lean on ethtool flow steering or reduce the NIC to a single queue.

Busy-Polling and need_wakeup — Driving the Rings Efficiently

Two mechanisms control when the kernel does work on behalf of the socket, and getting them right is what separates a fast AF_XDP app from a slow one.

XDP_USE_NEED_WAKEUP (a bind flag) adds a need_wakeup flag, visible in the FILL and TX rings (the rings userspace produces to). When set, the kernel sets XDP_RING_NEED_WAKEUP only when it actually needs userspace to wake it with a syscall; when clear, no syscall is needed. The win: if the application and the driver run on the same core, the kernel can yield to userspace instead of spinning, and on the TX path it avoids a sendto() per batch. The idiom from the docs:

if (xsk_ring_prod__needs_wakeup(&my_tx_ring))
    sendto(xsk_socket__fd(xsk_handle), NULL, 0, MSG_DONTWAIT, NULL, 0);

i.e. only call sendto() when the flag is actually set. The docs recommend always enabling this mode “as it usually leads to better performance.” On the RX side, when the kernel runs out of FILL-ring buffers it turns off interrupts (the NIC has nowhere to put packets), sets need_wakeup on the FILL ring, and the application must replenish the FILL ring and call poll() to restart reception.

Busy-polling integrates AF_XDP with the kernel’s SO_BUSY_POLL / SO_PREFER_BUSY_POLL mechanism. With busy-poll enabled, recvmsg/sendmsg/poll on the socket drive a sk_busy_loop() that runs the driver’s NAPI poll inline in the syscall, on the application’s CPU, so RX/TX progresses without waiting for a separate softirq context. In net/xdp/xsk.c, __xsk_recvmsg() does if (sk_can_busy_loop(sk)) sk_busy_loop(sk, 1); before returning. Note that AF_XDP sockets are effectively non-blocking: recvmsg/sendmsg with a blocking flag (without MSG_DONTWAIT) return -EOPNOTSUPP — the code path if (unlikely(need_wait)) return -EOPNOTSUPP; enforces this, so AF_XDP applications always use MSG_DONTWAIT and poll.

Setup With libbpf / libxdp — The Practical Path

Almost nobody drives the raw UAPI; the recommended path is libxdp (the AF_XDP helpers, historically in libbpf’s xsk.h, now maintained in the xdp-tools/libxdp project at xdp-project/xdp-tools). The docs are explicit: “We recommend that you use this library unless you have become a power user.” A minimal RX setup looks like:

/* 1. Allocate UMEM memory yourself (page-aligned). */
void *umem_area;
posix_memalign(&umem_area, getpagesize(), NUM_FRAMES * FRAME_SIZE);
 
/* 2. Create the UMEM + its FILL and COMPLETION rings. */
struct xsk_umem *umem;
struct xsk_ring_prod fq;   /* FILL  */
struct xsk_ring_cons cq;   /* COMPLETION */
xsk_umem__create(&umem, umem_area, NUM_FRAMES * FRAME_SIZE, &fq, &cq, NULL);
 
/* 3. Create the socket + its RX and TX rings, bound to ifname/queue. */
struct xsk_socket *xsk;
struct xsk_ring_cons rx;   /* RX */
struct xsk_ring_prod tx;   /* TX */
xsk_socket__create(&xsk, "eth0", /*queue=*/0, umem, &rx, &tx, NULL);
/* libxdp auto-loads a built-in XDP redirect program and inserts the
   socket into an XSKMAP for you, unless you pass
   XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD and load your own. */
 
/* 4. Prime the FILL ring so the kernel has frames to receive into. */
__u32 idx;
xsk_ring_prod__reserve(&fq, NUM_FRAMES, &idx);
for (int i = 0; i < NUM_FRAMES; i++)
    *xsk_ring_prod__fill_addr(&fq, idx++) = i * FRAME_SIZE;
xsk_ring_prod__submit(&fq, NUM_FRAMES);

The receive loop then peeks the RX ring, processes each descriptor, and recycles the frame back onto the FILL ring — the exact pattern from the docs’ multi-buffer example:

int rcvd = xsk_ring_cons__peek(&rx, BATCH, &idx_rx);
xsk_ring_prod__reserve(&fq, rcvd, &idx_fq);            /* refill as we consume */
for (int i = 0; i < rcvd; i++) {
    struct xdp_desc *desc = xsk_ring_cons__rx_desc(&rx, idx_rx++);
    char *pkt = xsk_umem__get_data(umem_area, desc->addr);
    process(pkt, desc->len);
    *xsk_ring_prod__fill_addr(&fq, idx_fq++) = desc->addr;  /* return the tray */
}
xsk_ring_prod__submit(&fq, rcvd);
xsk_ring_cons__release(&rx, rcvd);                     /* tell kernel we're done */

The xsk_ring_*__peek/reserve/submit/release calls are the lock-free producer/consumer primitives with the right memory barriers baked in — using them instead of hand-rolling the head/tail arithmetic is the whole point of the library.

Shared UMEM and Multi-Buffer

A single UMEM can be shared by multiple sockets via the XDP_SHARED_UMEM bind flag — across queues and even across netdevs. Each socket still has its own RX/TX rings, but only one FILL/COMPLETION pair exists per unique (netdev, queue id) tuple, and “It is the responsibility of a single process to handle the UMEM” because the rings are single-producer/single-consumer with no kernel-side locking. The XDP program decides which socket each packet lands on by choosing the XSKMAP index. This is how you scale AF_XDP across cores: one socket per RX queue, all sharing one UMEM, with the NIC’s RSS hashing steering flows to queues.

Multi-buffer support (the XDP_USE_SG bind flag, since the socket must opt in) lets a packet span several frames — a 9K jumbo frame as three 4K frames, or a header/payload split. Each descriptor refers to one frame, and the XDP_PKT_CONTD bit in desc->options signals “the packet continues in the next descriptor”; the last frame has it clear. Without XDP_USE_SG, multi-buffer packets are dropped. The XDP program must also be in multi-buffer mode (section name xdp.frags). For copy mode the frame limit is CONFIG_MAX_SKB_FRAGS + 1 (portably, cap at 18); for zero-copy it is whatever the NIC supports, queryable via the netlink NETDEV_A_DEV_XDP_ZC_MAX_SEGS attribute.

Failure Modes and Common Misunderstandings

“I see no traffic on my socket.” The number-one cause, straight from the docs’ FAQ: you bound to one queue but the NIC is spreading traffic across all queues by RSS. A physical NIC allocates one RX/TX queue pair per core, so binding to queue 0 on an 8-core box misses queues 1–7. Fix with ethtool -L <if> combined 1 (force a single queue) or steer your flow to the bound queue with ethtool -N.

“Packets are corrupted.” You fed the same UMEM frame into more than one ring at once — e.g. the FILL ring and the TX ring simultaneously — so the NIC was receiving into the frame while transmitting out of it. The docs warn explicitly: “Care has to be taken not to feed the same buffer in the UMEM into more than one ring at the same time.” Each frame must be owned by exactly one side at a time.

“Receive just stops.” You stopped replenishing the FILL ring. With no FILL buffers, the kernel has nowhere to put packets, disables interrupts, and (with need_wakeup) waits for you to refill and poll(). The RX ring is driven by the FILL ring; they must be kept in balance.

“bind() fails with XDP_ZEROCOPY.” The driver doesn’t support zero-copy for AF_XDP. Either drop the forced flag and accept copy-mode fallback, or use a driver/NIC that implements the XSK pool. Forcing a mode the driver can’t provide is a hard failure by design.

Uncertain

Verify: the exact set of in-tree drivers implementing AF_XDP zero-copy (the MEM_TYPE_XSK_BUFF_POOL / XDP_SETUP_XSK_POOL path) as of 6.12 and 6.18 LTS, and the per-driver zero-copy multi-buffer (NETDEV_A_DEV_XDP_ZC_MAX_SEGS) limits. Reason: the named drivers (i40e/ice/ixgbe/mlx5/stmmac) are from general knowledge and the v6.12 doc’s framing, not an exhaustive tree audit; zero-copy support is added driver-by-driver and shifts release-to-release. To resolve: grep the v6.12/v6.18 trees for xsk_buff_pool / XDP_SETUP_XSK_POOL handlers per driver, and consult the xdp-project driver feature matrix at the targeted LTS. uncertain

Alternatives and When to Choose Them

  • Plain XDP keeps processing in the kernel. If your packet logic (drop, forward, load-balance) fits in a verified eBPF program, you never need userspace at all — XDP is simpler and avoids the ring bookkeeping. Reach for AF_XDP only when the processing is too complex for the verifier, needs a userspace TCP/IP stack, or wants to live outside the kernel. AF_XDP is XDP plus a userspace destination.
  • DPDK is the incumbent userspace fast-packet framework. It binds the NIC to a userspace poll-mode driver, taking the interface away from the kernel entirely — no ping, no ssh, no kernel tooling on that NIC, and you must use DPDK’s own drivers. AF_XDP keeps the NIC a normal kernel netdev (the kernel still owns it; XDP just siphons selected traffic), at some cost in peak packets-per-second versus full DPDK. DPDK even ships an af_xdp PMD so you can run DPDK on top of AF_XDP, getting kernel-friendliness with the DPDK API. Choose DPDK for absolute maximum throughput on dedicated NICs; choose AF_XDP for high throughput while keeping the interface manageable.
  • The normal socket path (AF_INET + recv()/epoll) is what you want for ordinary applications — TCP, UDP, the full stack, kernel offloads, conntrack. AF_XDP gives up all of that (no TCP, no routing, no netfilter — you get raw frames) in exchange for speed. It is a specialist tool for packet-processing dataplanes, not a general socket.
  • io_uring is the other modern high-performance socket model — but it accelerates the normal stack’s syscalls (batched async recv/send/accept), not a stack bypass. io_uring makes the kernel stack faster to call; AF_XDP skips the kernel stack. Different regimes, same socket file descriptor world.

Production Notes

AF_XDP is the foundation of several userspace dataplanes that want DPDK-like speed without DPDK’s NIC monopoly. DPDK’s af_xdp poll-mode driver lets existing DPDK applications run on AF_XDP, a common deployment in environments that can’t dedicate a NIC. OVS (Open vSwitch) has an AF_XDP datapath (“userspace datapath with AF_XDP netdev”) for accelerated virtual switching. Cilium can use AF_XDP for certain high-throughput paths, and the xdp-project maintains libxdp, xdpdump, and the widely-used xdp-tutorial that most production AF_XDP code descends from. The original design and performance analysis is the Linux Plumbers 2018 paper by Björn Töpel and Magnus Karlsson (the AF_XDP authors), and Jonathan Corbet’s LWN article “Accelerating networking with AF_XDP” remains the best narrative introduction.

The operational lessons are consistent: zero-copy mode is dramatically faster but driver-dependent, so production setups query and assert the mode rather than assuming it; the FILL/COMPLETION ring balance is the most common source of stalls; and CPU pinning matters enormously — running the application thread on the same core as the NIC’s IRQ, with busy-poll and need_wakeup, is the configuration that hits line rate, while a mismatched core assignment can halve throughput.

See Also

Graph View

Backlinks