The Kernel Futex Interface

A futex (“fast userspace mutex”) is the kernel-side substrate that almost every userspace blocking lock on Linux — pthread_mutex_t, pthread_cond_t, glibc semaphores, the Go runtime’s thread parking — is built on. Its defining idea, due to Hubertus Franke, Rusty Russell, and Matthew Kirkwood (OLS 2002, “Fuss, Futexes and Furwocks”), is that the uncontended lock and unlock operations happen entirely in userspace as a single atomic read-modify-write on a shared 32-bit word — “in the uncontended case, no system calls are made, and no kernel involvement is necessary.” The kernel is entered through the futex() syscall only on contention: to block a thread whose lock attempt failed (FUTEX_WAIT), or to wake a blocked thread when the lock is released (FUTEX_WAKE). The kernel keeps no per-lock object at all in the common case — it materializes wait state lazily, in a global hash table keyed by the physical identity of the futex word, only while a thread is actually parked. This note covers the in-kernel mechanism and the two synchronization-correctness variants — priority-inheritance (PI) futexes and robust futexes — pinned to Linux 6.12, which is this vault’s standard pin and one of five longterm branches as of 2026-09-12 (kernel.org/releases.json lists 6.18.51, 6.12.109, 6.6.156, 6.1.187 and 5.15.220 as longterm, with 7.2.5 the latest stable). Every source quotation below is read at the v6.12 tag; every runtime measurement below was taken on this host’s running kernel, 7.1.13-200.fc44.x86_64, a stable release. Where the two disagree — and in one case, futex2’s NUMA flag, they do — that is called out explicitly. The userspace-facing futex() syscall API as an IPC surface is the subject of the Linux IPC MOC’s futex note; the generic/Go-runtime angle lives in Futex and OS Synchronization Primitives; this note is deliberately scoped to the kernel internals and the sync variants and cross-links rather than re-derives those siblings.

---

Mental Model — A Lock That Doesn’t Exist Until It’s Contended

The mental shift that makes futexes click is that there is no kernel lock object. A traditional kernel-mediated lock (a SysV semaphore, say) is a kernel data structure you create, reference by an ID, and destroy. A futex is none of that: it is just a u32 of ordinary user memory that userspace agrees, by convention, to treat as a lock word. The kernel does not know that word exists until a thread comes to it saying “I read this word, it had value V, and based on V I need to sleep — FUTEX_WAIT(uaddr, V).” Only then does the kernel create a transient wait-queue entry, and it tears that entry down the moment the thread is woken. Between contended episodes, the lock costs the kernel exactly nothing.

This is why the fast path is free. A userspace mutex unlock that has no waiters is a single atomic store (or compare-and-swap) to the word — no syscall, no kernel transition, no scheduler involvement. The expensive futex() syscall is paid only when a thread genuinely must block or genuinely must wake someone. The art of the futex protocol, then, is the userspace/kernel handshake that ensures a waiter and a waker never miss each other across that boundary — the classic lost-wakeup race.

flowchart TB
  subgraph US["Userspace (no kernel involvement)"]
    L["lock(): CAS word 0->1<br/>uncontended: done"]
    U["unlock(): word = 0<br/>uncontended: done"]
  end
  subgraph K["Kernel — only on contention"]
    HT["futex hash table<br/>jhash(key) -> bucket"]
    HB["futex_hash_bucket<br/>spinlock + plist of waiters"]
    W["FUTEX_WAIT:<br/>lock bucket, re-read word,<br/>if still == val: enqueue + schedule()"]
    WK["FUTEX_WAKE:<br/>lock bucket, walk plist,<br/>wake_up_q the matches"]
    HT --> HB
    HB --> W
    HB --> WK
  end
  L -.->|"CAS fails (contended)<br/>FUTEX_WAIT syscall"| W
  U -.->|"saw waiters bit<br/>FUTEX_WAKE syscall"| WK

The two-level futex architecture. What it shows: the lock/unlock fast paths are pure userspace atomics on the shared word; the kernel is reached only when an atomic op observes contention, at which point a global hash table maps the futex’s identity to a per-bucket lock and waiter list. The insight to take: the kernel holds no state at all for an uncontended futex — wait state is allocated lazily on the slow path and freed on wakeup, which is precisely what makes the common case cost zero syscalls.

---

The Central Mechanism — The Fast Path Never Enters the Kernel

Everything else in this note is a consequence of one design decision, so it is worth stating on its own and then measuring it: an uncontended lock and an uncontended unlock execute zero system calls. The futex() syscall is not “the lock”; it is the escape hatch the lock falls into when the atomic instruction discovers that somebody else is already there. If you internalize nothing else, internalize that the syscall count of a futex-backed mutex is a contention counter, not an operation counter.

Ulrich Drepper’s “Futexes Are Tricky” (Red Hat, version 1.6, dated 2011-11-05 — the revision read for this note, akkadia.org/drepper/futex.pdf) gives the canonical three-state mutex that realizes this. The word takes exactly three values, and the meaning of each is a userspace convention the kernel never sees:

Value	Meaning	Who may touch the kernel
0	unlocked	nobody — unlock of a 0→1 lock is a plain store
1	locked, no waiter is parked in the kernel	nobody — unlock is a lone atomic_dec
2	locked, and at least one waiter may be parked	unlock must issue FUTEX_WAKE

The 2 state is the whole trick. It is the userspace-maintained “there might be someone in the kernel” bit, and it is what lets the unlock path decide without asking the kernel whether a syscall is needed. Drepper’s mutex3 (the IA-32/AMD64 variant, which uses an atomic xchg) is:

/* Drepper, "Futexes Are Tricky" v1.6 §6, transcribed to C11 atomics. */
static _Atomic int val = 0;
 
static void m_lock(void) {
    int c = 0;
    /* Fast path: 0 -> 1. On success we are done: NO syscall. */
    if (!atomic_compare_exchange_strong(&val, &c, 1)) {   /* c receives the old value */
        /* Slow path. Announce "a waiter exists" by forcing the word to 2. */
        if (c != 2) c = atomic_exchange(&val, 2);
        while (c != 0) {                 /* someone still holds it */
            fwait(&val, 2);              /* FUTEX_WAIT(&val, 2) -- syscall */
            c = atomic_exchange(&val, 2);/* retry, re-asserting the waiters state */
        }
    }
}
static void m_unlock(void) {
    /* Fast path: 1 -> 0. atomic_fetch_sub returns the OLD value. */
    if (atomic_fetch_sub(&val, 1) != 1) { /* old != 1 means old == 2: waiters */
        atomic_store(&val, 0);
        fwake(&val, 1);                   /* FUTEX_WAKE(&val, 1) -- syscall */
    }
}

Line by line: m_lock’s first compare_exchange is the entire uncontended path — one atomic instruction, and c is left holding whatever was actually there. If the word was not 0, we unconditionally xchg it to 2; the return value tells us whether the lock became free in the meantime (c == 0, take it and leave) or is still held (c != 0, sleep). Crucially, FUTEX_WAIT is called with the value 2, the value we just wrote — that is the value whose continued presence means “I still must sleep”, and it is exactly what the kernel re-checks. In m_unlock, atomic_fetch_sub returns the old value: 1 means we were the only participant and the decrement already released the lock, so we return without a syscall. Anything else means the word was 2 — a waiter announced itself — so we must store 0 and pay for one FUTEX_WAKE.

Drepper tabulates the costs twice, and read together the two tables say something sharper than either alone. For the uncontended case (v1.6 §5), mutex2 costs 1 atomic operation and 0 futex syscalls to lock, and 1 atomic operation and 0 futex syscalls to unlock — against the naive mutex of §4, which needed 0 atomic ops but 1 syscall on every unlock. Drepper’s own summary of that trade: “We traded one system call for an atomic operation. This is almost always a good choice, especially here since the futex system call needs atomic instructions itself.” For the contended case (v1.6 §6) the comparison is mutex2 vs mutex3:

Contended cost	mutex2	mutex3
lock — atomic ops	3 + 2	2 + 1
lock — futex syscalls	1 + 1	1 + 1
unlock — atomic ops	1	1
unlock — futex syscalls	1	1

The notation a + b counts the fast-path cost plus the per-retry cost around the wait loop. Note what mutex3 does not buy: the syscall counts are identical. Its entire gain is in atomic operations, by replacing a second cmpxchg with an unconditional xchg — which matters because, as Drepper puts it, “The difference between executing one or two atomic operations on the same memory location on multiple processors at the same time is big.” The syscall count is set by the protocol, not by the instruction selection.

Measured: zero syscalls uncontended, ~0.25 per operation contended

The implementation above was compiled with gcc -O2 -pthread on this host (kernel 7.1.13-200.fc44.x86_64, AMD Ryzen AI MAX+ 395, 32 logical CPUs) with a counter around each raw syscall(SYS_futex, …):

$ ./futexdemo 1 200000     # 1 thread, 200 000 lock/unlock pairs
threads=1 iters=200000 counter=200000 FUTEX_WAIT=0 FUTEX_WAKE=0 total=0

$ ./futexdemo 2 200000     # 2 threads, 400 000 lock/unlock pairs
threads=2 iters=200000 counter=400000 FUTEX_WAIT=32809 FUTEX_WAKE=69108 total=101917

$ ./futexdemo 8 200000     # 8 threads, 1 600 000 lock/unlock pairs
threads=8 iters=200000 counter=1600000 FUTEX_WAIT=104012 FUTEX_WAKE=146210 total=250222

Two hundred thousand lock/unlock pairs, zero system calls. Add a second thread and 400,000 pairs cost 101,917 syscalls — roughly one futex entry per four critical sections. Note that FUTEX_WAKE outnumbers FUTEX_WAIT about two to one: the 2 state is conservative. Once any thread writes 2, every subsequent unlocker must wake, even if the waiter has already given up and re-acquired the lock. That over-waking is the price of keeping the unlock decision in userspace, and it is why FUTEX_WAKE on an empty hash bucket is optimized into a lock-free counter read (see below).

Because strace is not installed on this host, the syscall counts were independently confirmed with gdb -batch and a catch syscall futex catchpoint that counts and continues. catch syscall stops twice per call — once on entry and once on return — so the raw stop count is doubled, and the register values at the stop give the operation and the address:

$ gdb -batch -x trace2.gdb ./futexdemo      # run 2 60000
--- op histogram (each syscall appears twice: entry+exit) ---
     52 op=128
     94 op=129
    100 op=265
--- uaddr histogram ---
    146 uaddr=0x403048
    100 uaddr=0x7ffff7d9cce8
threads=2 iters=60000 counter=120000 FUTEX_WAIT=26 FUTEX_WAKE=47 total=73
ADDR_OF_val=0x403048

Decoding it: 0x403048 is the program’s own val — gdb printed &val at the end to confirm. It accounts for 146 stops = 73 syscalls, exactly the program’s internal count of 73. Split by operation, op=128 decodes as FUTEX_WAIT (0) | FUTEX_PRIVATE_FLAG (128) and op=129 as FUTEX_WAKE (1) | FUTEX_PRIVATE_FLAG (128), giving 26 waits and 47 wakes — again exactly the internal counters. The remaining 100 stops are on a libc address with op=265 = FUTEX_WAIT_BITSET (9) | FUTEX_CLOCK_REALTIME (256), and a backtrace shows them coming from glibc’s __futex_abstimed_wait_cancelable64 inside pthread_join — not from the mutex at all.

Measurement artifact, not a property of the mechanism

An earlier run of the contended program under gdb with only 3,000 iterations per thread reported zero mutex futex syscalls, because ptrace-stopping the process on every catchpoint serialized the two threads so thoroughly that they never actually collided. The mutex did not change; the sampling did. Any “the counter was zero” conclusion about a contention-driven mechanism must state the load under which it was measured. uncertain

sequenceDiagram
  autonumber
  participant T as Thread
  participant M as futex word in user memory
  participant K as Kernel
  Note over T,K: UNCONTENDED — the kernel is never entered
  T->>M: CAS 0 -> 1 (succeeds)
  Note right of M: word = 1 (locked, no waiters)
  T->>T: critical section
  T->>M: atomic_fetch_sub, old value was 1
  Note right of M: word = 0, old == 1 so no waiters
  Note over K: zero syscalls, zero kernel state

The uncontended lock/unlock path. What it shows: two atomic read-modify-write instructions on one word of ordinary user memory, and nothing else — no futex() call, no wait queue, no scheduler involvement. The insight to take: the kernel is not merely fast here, it is absent, which is why the measured syscall count for 200,000 uncontended pairs is exactly 0.

sequenceDiagram
  autonumber
  participant A as Thread A (holder)
  participant M as futex word
  participant K as Kernel (hash bucket)
  participant B as Thread B (contender)
  A->>M: CAS 0 -> 1 succeeds, enters critical section
  B->>M: CAS 0 -> 1 fails, observes 1
  B->>M: xchg -> 2 (announce "a waiter exists")
  B->>K: futex(uaddr, FUTEX_WAIT_PRIVATE, 2)
  K->>M: re-read word under bucket lock
  alt word still == 2
    K->>K: enqueue futex_q, schedule()
  else word changed
    K-->>B: -EAGAIN, do not sleep
  end
  A->>M: atomic_fetch_sub, old value was 2
  Note right of M: old != 1 so a waiter was announced
  A->>M: store 0
  A->>K: futex(uaddr, FUTEX_WAKE_PRIVATE, 1)
  K->>K: walk plist, futex_wake_mark, wake_up_q after unlock
  K-->>B: return 0
  B->>M: xchg -> 2, retry the acquire

The contended path, and the only path that costs a system call. What it shows: the transition from value 1 to value 2 is what obliges the holder to make a FUTEX_WAKE call it would otherwise skip, and the kernel’s re-read of the word under the bucket lock is what turns a lost wakeup into a harmless -EAGAIN. The insight to take: userspace, not the kernel, decides whether a syscall is needed — the 2 state is a message from the contender to the future unlocker, delivered through memory rather than through the kernel.

---

Mechanical Walk-through — Hash Table, Keys, Wait and Wake

Identifying a futex: the key and the hash bucket

Because a futex is just user memory, the kernel must derive a stable identity for the word so that a waiter and a waker who touch the same lock land on the same wait queue — even across processes sharing memory. That identity is the union futex_key, computed by get_futex_key() in core.c. The kernel distinguishes two cases (core.c, v6.12):

• Private futexes (the FUTEX_PRIVATE_FLAG is set, meaning the futex is only used within one address space). The key is simply (current->mm, address, 0) — the mm pointer plus the virtual address. No page lookup is needed: “As the mm cannot disappear under us and the ‘key’ only needs virtual address, we dont even have to find the underlying vma.” This is the fast, common case for an ordinary process-local pthread_mutex_t.
• Shared futexes (no private flag; the futex word lives in memory shared between processes via mmap/shmem). The key must be independent of any one process’s virtual address, so it is (inode->i_sequence, page->index, offset_within_page) — anchored to the physical backing object. get_futex_key() calls get_user_pages_fast() to pin the page and resolve its mapping, then derives a machine-wide-unique inode sequence number via get_inode_sequence_number().

The key is then hashed into the global table by futex_hash():

struct futex_hash_bucket *futex_hash(union futex_key *key)
{
	u32 hash = jhash2((u32 *)key, offsetof(typeof(*key), both.offset) / 4,
			  key->both.offset);
	return &futex_queues[hash & (futex_hashsize - 1)];
}

jhash2 is Bob Jenkins’ hash over the key words; the result is masked into the power-of-two-sized futex_queues array. The table is global and sized at boot proportional to the number of CPUs — futex_init() in core.c computes futex_hashsize = roundup_pow_of_two(256 * num_possible_cpus()) and then hands that to alloc_large_system_hash(), which may round it down under memory pressure before futex_hashsize is reset from the returned shift. On the 32-CPU host used for the measurements in this note the requested size is therefore 256 × 32 = 8192 buckets. Multiple distinct futexes can collide into one bucket — that is fine, because each waiter records its own futex_key and the wake path matches on the full key, not on the bucket.

flowchart TD
  A["userspace calls futex(uaddr, op, ...)"] --> B{"FUTEX_PRIVATE_FLAG<br/>(bit 7, value 128) set?"}
  B -- "yes: private" --> C["key = (current->mm, uaddr, 0)<br/>no page walk, no GUP<br/>offset bit FUT_OFF_MMSHARED clear"]
  B -- "no: shared" --> D["get_user_pages_fast(uaddr)<br/>pin the backing folio"]
  D --> E{"folio_test_anon(folio)?"}
  E -- "no: file or shmem backed" --> F["key = (inode->i_sequence,<br/>folio index, offset in page)<br/>offset |= FUT_OFF_INODE (1)<br/>MAP_SHARED|MAP_ANONYMOUS lands here:<br/>it is shmem, not anon"]
  E -- "yes: anonymous folio" --> G["key = (mm, address, offset)<br/>offset |= FUT_OFF_MMSHARED (2)<br/>a read-only anon page is rejected<br/>with -EFAULT: it can never change"]
  C --> H["futex_hash(key):<br/>jhash2 over the key words,<br/>seeded with key->both.offset"]
  F --> H
  G --> H
  H --> I["bucket = &futex_queues[hash & (futex_hashsize - 1)]<br/>futex_hashsize = roundup_pow_of_two(256 * nr_cpus)"]
  I --> J["spin_lock(&hb->lock)"]
  J --> K["walk hb->chain (a plist)<br/>futex_match(&this->key, &key)<br/>on the FULL key, not the bucket"]

How an address becomes a wait queue. What it shows: the private/shared decision is made by one bit of the op argument and determines whether the kernel can key off (mm, virtual address) or must pin the page and key off the physical backing object; either way the key is hashed into one global table whose buckets are shared by unrelated futexes. The insight to take: a hash collision is harmless because futex_match() compares the whole key, but a private/shared mismatch is not — it produces two different keys for the same memory, so a private waiter and a shared waker land in different buckets and never meet.

A futex_hash_bucket is deliberately tiny and cache-line aligned (futex.h, v6.12):

struct futex_hash_bucket {
	atomic_t waiters;
	spinlock_t lock;
	struct plist_head chain;
} ____cacheline_aligned_in_smp;

The lock serializes all operations on the bucket; chain is a priority-sorted list (plist) of waiters, so that a wake can prefer the highest-priority waiter (important for real-time correctness); and waiters is an atomic counter used by the lock-free wakeup-skip optimization described below. Each parked task is represented by a struct futex_q linked into chain, carrying its task, its key, the lock_ptr of the bucket, a wake handler, and a bitset.

FUTEX_WAIT: verify-then-sleep, under the bucket lock

The single most important invariant of FUTEX_WAIT is that the kernel re-reads the futex word under the bucket lock and checks it still equals the caller’s expected value before sleeping. Without this, a wakeup that happened between userspace’s read and the syscall would be lost forever. The path is futex_wait() → __futex_wait() → futex_wait_setup() then futex_wait_queue() (waitwake.c, v6.12).

futex_wait_setup() resolves the key, locks the bucket, and reads the current word uval:

ret = get_futex_key(uaddr, flags, &q->key, FUTEX_READ);
...
*hb = futex_q_lock(q);
ret = futex_get_value_locked(&uval, uaddr);
...
if (uval != val) {
	futex_q_unlock(*hb);
	ret = -EWOULDBLOCK;
}

If the word no longer equals the value userspace passed in (val), the lock was released or grabbed by someone else in the interim, so the kernel returns -EWOULDBLOCK (which surfaces to userspace as EAGAIN) without sleeping — userspace retries its atomic. The source comment states the contract exactly: “The basic logical guarantee of a futex is that it blocks ONLY if cond(var) is known to be true at the time of blocking… If we locked the hash-bucket after testing *uaddr, that would open a race condition where we could block indefinitely with cond(var) false.” Locking the bucket before the final read is what closes that window.

If the word matches, futex_wait_queue() enqueues the futex_q and sleeps:

set_current_state(TASK_INTERRUPTIBLE|TASK_FREEZABLE);
futex_queue(q, hb);   /* enqueue then release hb->lock */
if (timeout) hrtimer_sleeper_start_expires(timeout, HRTIMER_MODE_ABS);
if (likely(!plist_node_empty(&q->list))) {
	if (!timeout || timeout->task)
		schedule();
}
__set_current_state(TASK_RUNNING);

The task state is set to TASK_INTERRUPTIBLE before enqueueing and releasing the bucket lock, so any concurrent waker that takes the bucket lock after this point will find the task on the list and its state already set — no wakeup can slip through. After schedule() returns (woken, timed out, or signalled), __futex_wait() calls futex_unqueue(); if the task was not still on the list, it was woken legitimately and returns success.

FUTEX_WAKE: walk the bucket, mark, wake outside the lock

futex_wake() resolves the key, locks the bucket, walks chain, and for each futex_q whose key matches it invokes the wake handler:

plist_for_each_entry_safe(this, next, &hb->chain, list) {
	if (futex_match(&this->key, &key)) {
		if (this->pi_state || this->rt_waiter) { ret = -EINVAL; break; }
		if (!(this->bitset & bitset)) continue;
		this->wake(&wake_q, this);
		if (++ret >= nr_wake) break;
	}
}
spin_unlock(&hb->lock);
wake_up_q(&wake_q);

Two subtleties: the wake handler (futex_wake_mark) does not call wake_up() directly — it removes the futex_q from the list and adds the task to a wake_q, and only wake_up_q() after the bucket lock is dropped actually makes tasks runnable, to keep the bucket lock hold-time minimal. And nr_wake (the syscall’s val argument) bounds how many are woken — a plain mutex unlock wakes one; a condition-variable broadcast might wake INT_MAX.

Closing the lost-wakeup race with the waiters counter

There is a lock-free fast path on the wake side: if no thread is parked, the waker can skip taking the bucket lock entirely (futex_hb_waiters_pending() reads the atomic waiters count and returns early when it is zero). This optimization is dangerous unless it is impossible for a waker to both miss the futex-value change and miss the enqueue. The kernel proves it cannot, with a pair of memory barriers documented at the top of waitwake.c. The waiter does waiters++ (barrier A) then reads the futex word; the waker writes the futex word then (barrier B) reads waiters. This is a Dekker-style store-load pattern:

X = Y = 0
w[X]=1     w[Y]=1
MB         MB
r[Y]=y     r[X]=x

The barriers guarantee x==0 && y==0 is impossible — i.e. the waker cannot simultaneously read “no waiters” and have its value-change be missed by the waiter. Either the waiter sees the new value (and bails out of FUTEX_WAIT with -EWOULDBLOCK), or the waker sees a nonzero waiter count (and takes the bucket lock to wake it). This is the formal heart of futex correctness and connects directly to the kernel memory barriers and the LKMM.

---

The Core and Bitset Operations

The futex() syscall multiplexes operations through its op argument, dispatched by do_futex() in syscalls.c. Its prototype is fixed at six arguments regardless of which operation is selected, and each operation reinterprets the unused slots differently — which is the single largest source of confusion when reading futex code:

long syscall(SYS_futex, u32 *uaddr, int op, u32 val,
             const struct timespec *utime, u32 *uaddr2, u32 val3);

The following table is transcribed directly from the #defines in include/uapi/linux/futex.h at v6.12 (lines 11–24) cross-referenced against the switch (cmd) in kernel/futex/syscalls.c at v6.12 (do_futex(), lines 84–130). ”—” means the argument is ignored for that operation.

Op	#	val	utime	uaddr2	val3	Semantics	Typical caller
FUTEX_WAIT	0	expected value	relative timeout	—	forced to FUTEX_BITSET_MATCH_ANY	block while *uaddr == val	pthread_mutex_lock slow path
FUTEX_WAKE	1	max waiters to wake	—	—	forced to MATCH_ANY	wake up to val waiters	pthread_mutex_unlock slow path
FUTEX_FD	2	—	—	—	—	removed: not in do_futex()’s switch, falls through to -ENOSYS	none — deleted in Linux 2.6.26
FUTEX_REQUEUE	3	nr_wake	reused as nr_requeue	target futex	—	wake val, move the rest to uaddr2, no value check	effectively deprecated, racy
FUTEX_CMP_REQUEUE	4	nr_wake	reused as nr_requeue	target futex	expected value of *uaddr	as above, but verify *uaddr == val3 first	historically pthread_cond_broadcast — no longer glibc (see below)
FUTEX_WAKE_OP	5	nr_wake on uaddr	reused as nr_wake2	second futex	encoded FUTEX_OP(op,oparg,cmp,cmparg)	atomically modify *uaddr2, wake on uaddr, conditionally wake on uaddr2	historically pthread_cond_signal — no longer glibc
FUTEX_LOCK_PI	6	—	absolute, forced CLOCK_REALTIME	—	—	PI-aware acquire, blocking	PTHREAD_PRIO_INHERIT mutex
FUTEX_UNLOCK_PI	7	—	—	—	—	PI-aware release, deboost + hand off	PTHREAD_PRIO_INHERIT mutex
FUTEX_TRYLOCK_PI	8	—	—	—	—	FUTEX_LOCK_PI with trylock=1, never blocks	pthread_mutex_trylock on a PI mutex
FUTEX_WAIT_BITSET	9	expected value	absolute, CLOCK_MONOTONIC unless FUTEX_CLOCK_REALTIME	—	32-bit wait bitset	block while *uaddr == val, wake only on an intersecting bitset	pthread_cond_timedwait, glibc rwlock
FUTEX_WAKE_BITSET	10	max waiters to wake	—	—	32-bit wake bitset	wake waiters whose bitset intersects val3	glibc rwlock reader/writer wakeups
FUTEX_WAIT_REQUEUE_PI	11	expected value	absolute	target PI futex	forced to MATCH_ANY	wait on a non-PI futex, expecting to be requeued onto the PI futex uaddr2	intended for pthread_cond_wait on a PI mutex — glibc 2.43 defines the macro but has no caller
FUTEX_CMP_REQUEUE_PI	12	nr_wake	reused as nr_requeue	target PI futex	expected value of *uaddr	requeue from a plain futex onto a PI futex, taking ownership on the way	intended for pthread_cond_signal on a PI mutex — same, macro without caller
FUTEX_LOCK_PI2	13	—	absolute, CLOCK_MONOTONIC unless FUTEX_CLOCK_REALTIME	—	—	FUTEX_LOCK_PI with a sane clock	newer PI callers wanting a monotonic timeout

Three things in that table repay a second look. First, the utime slot is only treated as a struct timespec * for the five commands listed by futex_cmd_has_timeout() — FUTEX_WAIT, FUTEX_LOCK_PI, FUTEX_LOCK_PI2, FUTEX_WAIT_BITSET, FUTEX_WAIT_REQUEUE_PI. For every other command SYSCALL_DEFINE6(futex, …) passes (unsigned long)utime straight through as the integer val2, so the pointer argument becomes a count. That is why FUTEX_CMP_REQUEUE’s nr_requeue is documented as living in the timeout position.

Second, FUTEX_LOCK_PI and FUTEX_LOCK_PI2 differ only in their clock. do_futex() sets flags |= FLAGS_CLOCKRT unconditionally for FUTEX_LOCK_PI and then falls through into the FUTEX_LOCK_PI2 case, so the older op is permanently pinned to CLOCK_REALTIME — a timeout that a settimeofday() or a leap second can move. FUTEX_LOCK_PI2 exists to give callers CLOCK_MONOTONIC, which is what almost everyone actually wants.

Third, FUTEX_WAIT is the odd one out on timeout interpretation. futex_init_timeout() converts its timeout from relative to absolute with ktime_add_safe(ktime_get(), *t), while every other timeout-bearing command takes an absolute time. A caller that passes an absolute deadline to FUTEX_WAIT will sleep for approximately the number of seconds since 1970.

The two modifier flags

The op argument is not a plain enum: its top bits carry two modifiers, extracted by futex_to_flags() and masked off with FUTEX_CMD_MASK = ~(FUTEX_PRIVATE_FLAG | FUTEX_CLOCK_REALTIME) before the switch.

Flag	Value	Sets	What it actually changes	Where it is legal
FUTEX_PRIVATE_FLAG	128 (0x80)	clears FLAGS_SHARED	get_futex_key() keys on (current->mm, address, 0) and skips get_user_pages_fast() entirely — no page pin, no inode lookup, no folio refcount. The futex becomes invisible to other address spaces.	every operation
FUTEX_CLOCK_REALTIME	256 (0x100)	sets FLAGS_CLOCKRT	the (absolute) timeout is measured against CLOCK_REALTIME instead of CLOCK_MONOTONIC	only FUTEX_WAIT_BITSET, FUTEX_WAIT_REQUEUE_PI, FUTEX_LOCK_PI2; anything else returns -ENOSYS

The FUTEX_CLOCK_REALTIME restriction is enforced by an explicit guard at the top of do_futex() and is easy to verify from userspace. Probing the running kernel (7.1.13) with raw syscall() calls:

futex(FUTEX_FD) -- removed op 2                ret=-1   errno=38  Function not implemented
futex(FUTEX_WAIT|CLOCK_REALTIME) op=256        ret=-1   errno=38  Function not implemented
futex(FUTEX_WAIT_BITSET|CLOCK_REALTIME|PRIV)   ret=-1   errno=11  Resource temporarily unavailable

FUTEX_WAIT | FUTEX_CLOCK_REALTIME is rejected with ENOSYS even though both constants are perfectly valid on their own — the combination is not. FUTEX_WAIT_BITSET | FUTEX_CLOCK_REALTIME | FUTEX_PRIVATE_FLAG is accepted and reaches the value check, returning EAGAIN because the probe deliberately passed a mismatched expected value. And FUTEX_FD, op 2, is still #defined in the UAPI header for ABI stability but has no case in the switch, so it falls off the end of do_futex() into return -ENOSYS: a constant that exists, compiles, and cannot work.

The private flag is the one that matters for performance, and glibc uses it aggressively — sysdeps/nptl/futex-internal.h at glibc 2.43 defines FUTEX_PRIVATE as LLL_PRIVATE and threads a private argument through every internal futex wrapper, selected per mutex by PTHREAD_MUTEX_PSHARED() (glibc 2.43 nptl/pthread_mutex_lock.c). Every measured op=128/op=129 in the gdb trace above is that mechanism at work.

The operations in prose

• FUTEX_WAIT (0) / FUTEX_WAKE (1) — the bread and butter described above. FUTEX_WAIT(uaddr, val) blocks while *uaddr == val; FUTEX_WAKE(uaddr, n) wakes up to n waiters on uaddr.
• FUTEX_WAIT_BITSET (9) / FUTEX_WAKE_BITSET (10) — a generalization carrying a 32-bit bitset. A waiter is only woken if its bitset intersects the waker’s bitset (this->bitset & bitset in the wake loop above). The plain FUTEX_WAIT/FUTEX_WAKE are simply the bitset variants with bitset == FUTEX_BITSET_MATCH_ANY (0xffffffff), which do_futex() substitutes via val3 = FUTEX_BITSET_MATCH_ANY. The bitset is how glibc implements a single wait-queue that distinguishes, e.g., reader vs writer wakeups without spurious wakeups. A second difference: FUTEX_WAIT_BITSET’s timeout is treated as an absolute time (and can use CLOCK_REALTIME via the FUTEX_CLOCK_REALTIME flag), whereas plain FUTEX_WAIT’s timeout is relative.
• FUTEX_WAKE_OP (5) — an atomic “modify second futex, then conditionally wake on both” operation (futex_atomic_op_inuser), used by glibc to fuse a condition-variable internal state update with a wake.

Requeue and the condition-variable thundering herd

FUTEX_REQUEUE (3) / FUTEX_CMP_REQUEUE (4) exist to fix a specific, real performance disaster in pthread_cond_broadcast(). A POSIX condition variable is, internally, two futexes: the condvar’s own word and the associated mutex. When pthread_cond_broadcast() must wake everyone waiting on the condvar, a naïve implementation would FUTEX_WAKE all of them — but every woken thread immediately tries to lock the same associated mutex, so all but one instantly block again on the mutex. That is the thundering herd: a storm of wakeups, context switches, and re-sleeps for threads that had no chance of making progress. The FUTEX_CMP_REQUEUE man page states the fix exactly (FUTEX_CMP_REQUEUE(2const)):

“If a waker thread used FUTEX_WAKE, then all waiters waiting on B would be woken up, and they would all try to acquire lock A. However, waking all of the threads in this manner would be pointless because all except one of the threads would immediately block on lock A again. By contrast, a requeue operation wakes just one waiter and moves the other waiters to lock A, and when the woken waiter unlocks A then the next waiter can proceed.”

The textbook motivation is historically true and currently false

Every account of FUTEX_CMP_REQUEUE, including the man page quoted above and Drepper v1.6 §4, explains it as the fix for pthread_cond_broadcast(). glibc no longer uses it. Checking the source at the tags: nptl/pthread_cond_broadcast.c at glibc 2.24 calls lll_futex_requeue() (7 occurrences of “requeue” in the file); at glibc 2.25 that file has zero occurrences and broadcasts with a plain futex_wake(&cond->__data.__g_signals[g1], INT_MAX, private). At glibc 2.43 it is still a plain futex_wake, pthread_cond_signal.c is a plain futex_wake(…, 1, …), and sysdeps/nptl/futex-internal.h provides no requeue wrapper at all. Torvald Riegel’s condition-variable rewrite in glibc 2.25 abandoned requeue-based hand-off in favour of a group/signal scheme. The lll_futex_requeue, lll_futex_wake_unlock (which issues FUTEX_WAKE_OP) and lll_futex_cmp_requeue_pi macros still exist in glibc 2.43’s sysdeps/nptl/lowlevellock-futex.h, but I could not find an in-tree caller for any of them. So requeue is best understood as a kernel facility whose original consumer has moved on — still supported, still used by other runtimes and by hand-written locks, no longer on glibc’s condvar path. uncertain (to resolve: git grep -n lll_futex_requeue over a full glibc 2.43 checkout, which a blob-by-blob fetch cannot substitute for)

Requeue therefore wakes nr_wake waiters (typically one) on the source futex and atomically moves (requeue_futex() in requeue.c) up to nr_requeue of the remaining waiters from the source bucket’s chain to the target futex’s bucket, without waking them. They wake naturally, one at a time, as the mutex is handed off. FUTEX_CMP_REQUEUE additionally takes an expected value val3 and verifies the source word still equals it under the bucket lock before requeueing — closing a race where the condvar state changed between userspace’s decision to broadcast and the syscall; FUTEX_REQUEUE (no compare) is the older, racier form and is effectively deprecated for this reason. The requeue acquires both buckets’ locks via double_lock_hb(), which orders the two locks by address to avoid an AB-BA deadlock (the kind lockdep would otherwise flag).

---

PI-Futexes — Priority Inheritance in the Word

A plain futex cannot solve priority inversion: if a low-priority thread holds a futex that a high-priority thread needs, and a medium-priority thread is runnable, the medium thread starves the holder and the high thread is blocked indefinitely. The cure is priority inheritance — temporarily boosting the holder to the waiter’s priority — and Linux exposes it to userspace (as pthread_mutexattr_setprotocol(PTHREAD_PRIO_INHERIT)) via PI-futexes: FUTEX_LOCK_PI (6), FUTEX_UNLOCK_PI (7), FUTEX_TRYLOCK_PI (8), and FUTEX_LOCK_PI2 (13, the absolute-CLOCK_MONOTONIC-timeout variant). The full PI protocol, the chain walk, and the rt_mutex machinery are the subject of Priority Inheritance and the RT-Mutex; here we cover only the futex-specific glue.

PI-futexes impose a defined layout on the futex word, unlike plain futexes where the value is opaque to the kernel. The word encodes the owner’s thread ID in its low 30 bits plus two flag bits (uapi/linux/futex.h):

#define FUTEX_WAITERS    0x80000000   /* contended: kernel state exists */
#define FUTEX_OWNER_DIED 0x40000000   /* owner died holding the lock    */
#define FUTEX_TID_MASK   0x3fffffff   /* the owner's TID                */

packet-beta
0: "W"
1: "D"
2-31: "FUTEX_TID_MASK — owner's thread ID, 30 bits"

The PI/robust futex word, 32 bits. Convention: position 0 here is bit 31 of the word — the diagram is drawn most-significant-bit first, the order in which the hex constants read. W is FUTEX_WAITERS (0x80000000, bit 31) and D is FUTEX_OWNER_DIED (0x40000000, bit 30). What it shows: the kernel imposes a real, documented layout on this word, unlike a plain futex where the value is opaque. The insight to take: the two flag bits are free real estate rather than a range compromise. PID_MAX_LIMIT in include/linux/threads.h at v6.12 is 4 * 1024 * 1024 = 2²² on 64-bit — and that header carries the cross-reference explicitly: “[NOTE: PID/TIDs are limited to 2^30 ~= 1 billion, see FUTEX_TID_MASK.]”. The host used for this note reports /proc/sys/kernel/pid_max = 4194304, exactly 2²², leaving eight spare bits of headroom. Crucially TID == 0 is not a valid owner, which is what lets 0 mean “unlocked” in the very same encoding.

The resulting state machine is small enough to tabulate, and every row below is a case explicitly handled by futex_lock_pi_atomic() in kernel/futex/pi.c at v6.12 (lines 525–608):

Word value	Meaning	What FUTEX_LOCK_PI does	Kernel state exists?
0x00000000	free, never contended	userspace CAS 0 → TID succeeds — no syscall	no
TID	held by TID, no waiters	first waiter CASes TID → TID|FUTEX_WAITERS, then attach_to_pi_owner()	not yet — created now
TID | FUTEX_WAITERS	held, at least one waiter parked	futex_top_waiter() finds an existing futex_q, so attach_to_pi_state() joins the existing pi_state	yes: a futex_pi_state with an embedded rt_mutex_base
caller's own TID (any flags)	the caller already holds it	-EDEADLK, immediately — self-deadlock detection is one comparison	irrelevant
FUTEX_OWNER_DIED (TID bits zero)	previous owner died, lock is free but state may be inconsistent	takes over: newval = (uval & FUTEX_OWNER_DIED) | vpid — the died bit is deliberately preserved so the new owner still learns about it	no
TID | FUTEX_OWNER_DIED | FUTEX_WAITERS	owner died while waiters were parked	handle_futex_death() set the bit on exit and woke a waiter, which then takes over and reports EOWNERDEAD	yes, until drained

One asymmetry surprises most readers: FUTEX_WAITERS is not cleared when the last waiter is handed the lock. wake_futex_pi() writes newval = FUTEX_WAITERS | task_pid_vnr(new_owner) unconditionally, with the comment “The WAITERS bit is always kept enabled while there is PI state around.” The consequence is that once a PI futex has been contended even once, every subsequent unlock by that owner is forced through FUTEX_UNLOCK_PI — the userspace TID → 0 CAS will keep failing — until the kernel-side pi_state is finally torn down. A PI mutex that has ever been contended is permanently more expensive than one that has not, and that is by design: the kernel must not lose track of who owns a lock it is running inheritance on.

The uncontended PI-lock fast path is still pure userspace: a CAS of 0 -> own_TID. This is visible verbatim in glibc: nptl/pthread_mutex_lock.c at glibc 2.43 does oldval = atomic_compare_and_exchange_val_acq (&mutex->__data.__lock, newval /* == id */, 0); and only on failure calls __futex_lock_pi64() — “The mutex is locked. The kernel will now take care of everything,” as the comment there puts it. When the returned word carries FUTEX_OWNER_DIED, glibc clears it with atomic_fetch_and_acquire (&mutex->__data.__lock, ~FUTEX_OWNER_DIED) and returns EOWNERDEAD to the application. On contention, the kernel side is futex_lock_pi_atomic() in pi.c (pi.c, v6.12). It reads the word, detects self-deadlock ((uval & FUTEX_TID_MASK) == vpid → -EDEADLK), and — being the first waiter — sets the waiters bit so the owner is forced into the kernel on unlock:

/* First waiter. Set the waiters bit before attaching ourself to the
 * owner. If owner tries to unlock, it will be forced into the kernel
 * and blocked on hb->lock. */
newval = uval | FUTEX_WAITERS;
ret = lock_pi_update_atomic(uaddr, uval, newval);
...
return attach_to_pi_owner(uaddr, newval, key, ps, exiting);

Setting FUTEX_WAITERS is the crux: once set, the owner’s userspace unlock CAS (own_TID -> 0) fails, forcing the owner into FUTEX_UNLOCK_PI where the kernel can perform the priority-deboost and hand the lock to the highest-priority waiter. The kernel then attaches a struct futex_pi_state — which embeds a real rt_mutex_base pi_mutex and tracks the owner task — and the waiting thread blocks on that rt_mutex via __rt_mutex_start_proxy_lock() + rt_mutex_wait_proxy_lock(). It is the rt_mutex, not the futex code, that runs the PI chain walk and boosts the owner. On FUTEX_UNLOCK_PI, wake_futex_pi() calls __rt_mutex_futex_unlock() to deboost and select the next owner, writing that owner’s TID into the userspace word. This is why a PI-futex word is kernel-readable: the kernel must know who owns the lock to run inheritance, whereas a plain futex word is meaningful only to userspace. PI-futexes invert the central mechanism of this note. A plain futex keeps the kernel ignorant on purpose so the fast path can skip it; a PI futex must tell the kernel the owner’s identity, because you cannot boost a thread you cannot name. The TID in the word is the price of admission for priority inheritance.

sequenceDiagram
  autonumber
  participant H as High-prio waiter
  participant W as futex word
  participant F as futex/pi.c
  participant R as rt_mutex (pi_state->pi_mutex)
  participant L as Low-prio owner
  L->>W: CAS 0 -> TID_low succeeds, no syscall
  Note over L: runs at low priority, holds the lock
  H->>W: CAS 0 -> TID_high fails, word is TID_low
  H->>F: futex(FUTEX_LOCK_PI)
  F->>F: uval TID bits != my vpid, so no -EDEADLK
  F->>F: futex_top_waiter() finds nothing, this is the first waiter
  F->>W: lock_pi_update_atomic, TID_low -> TID_low | FUTEX_WAITERS
  Note right of W: the owner's unlock CAS is now guaranteed to fail
  F->>F: attach_to_pi_owner() builds a futex_pi_state for TID_low
  F->>R: __rt_mutex_start_proxy_lock(), enqueue H as a waiter
  R->>L: PI chain walk boosts L to H's priority
  Note over L: L now runs at high priority and finishes fast
  L->>W: CAS TID_low -> 0 FAILS, the WAITERS bit is set
  L->>F: futex(FUTEX_UNLOCK_PI) -- forced into the kernel
  F->>R: __rt_mutex_futex_unlock(), deboost L, pick top waiter H
  F->>W: write FUTEX_WAITERS | TID_high
  Note right of W: WAITERS stays set while pi_state lives
  F-->>H: return 0, H owns the lock

Priority-inheritance boost propagation, from the futex side. What it shows: the futex code’s entire contribution is (a) setting FUTEX_WAITERS so the owner’s userspace unlock cannot succeed, (b) attaching a futex_pi_state that names the owner, and (c) writing the next owner’s TID back into user memory. The boost itself, step 11, is the rt_mutex’s job — see Priority Inheritance and the RT-Mutex. The insight to take: step 7 is the hinge of the whole protocol. Without the FUTEX_WAITERS bit the low-priority owner would release the lock in userspace, the kernel would never learn about it, and the deboost in step 14 would never happen — leaving a permanently over-prioritized thread.

---

Robust Futexes — Surviving a Crashed Lock-Holder

A second correctness variant addresses a different failure: what happens when a thread dies while holding a futex? With a plain mutex, a crash mid-critical-section leaves the lock word stuck at the dead owner’s value forever — every other thread blocks on it eternally. Robust futexes let the kernel clean up after a dead holder so waiters are released.

The mechanism is a per-task robust list. A thread registers a struct robust_list_head once via set_robust_list(2), and thereafter (entirely in userspace, glibc does this) links each futex it locks into a per-thread linked list whose head the kernel knows. The structures are part of the syscall ABI (uapi/linux/futex.h):

struct robust_list { struct robust_list __user *next; };
struct robust_list_head {
	struct robust_list list;            /* head; points to itself if empty */
	long futex_offset;                  /* offset from list node to the futex word */
	struct robust_list __user *list_op_pending;  /* the in-flight lock, race window */
};

The clever detail is futex_offset and list_op_pending. The list links live inside each userspace lock object, at a fixed offset (futex_offset) from the futex word — so the kernel can walk the list of lock objects and find each one’s futex word without hardcoding any layout. And because adding yourself to the list cannot be atomic with acquiring the lock, userspace first stores the to-be-taken lock into list_op_pending, then acquires, then links it and clears the pending field — so the kernel always sees a lock the thread might hold even mid-acquire.

On thread exit (including a crash that the kernel turns into do_exit()), futex_cleanup() in core.c checks tsk->robust_list and, if set, calls exit_robust_list(). This walks the list (very defensively — it is userspace memory and could be corrupt or circular, hence the ROBUST_LIST_LIMIT of 2048 and cond_resched()), and for each entry calls handle_futex_death():

owner = uval & FUTEX_TID_MASK;
if (owner != task_pid_vnr(curr))
	return 0;                         /* not actually ours; skip */
/* Set the OWNER_DIED bit atomically via cmpxchg, preserving WAITERS,
 * then wake a waiter so it can take over and notice OWNER_DIED. */
mval = (uval & FUTEX_WAITERS) | FUTEX_OWNER_DIED;
... futex_cmpxchg_value_locked(&nval, uaddr, uval, mval) ...

So a dead holder’s futexes get the FUTEX_OWNER_DIED bit set in their word, and any waiter is woken with futex_wake(). The woken waiter sees FUTEX_OWNER_DIED and knows the lock’s protected data may be inconsistent — glibc surfaces this to the application as EOWNERDEAD from pthread_mutex_lock(), and the application can either recover the state and call pthread_mutex_consistent() or mark the mutex permanently unusable. The kernel only validates the owner TID matches the dying task, so a malformed robust list can at worst leak a lock, never wake the wrong waiter. Note PI-futexes and robust futexes compose: a robust PI-futex hands ownership through the rt_mutex path while also setting OWNER_DIED.

---

futex2 — futex_waitv() for Waiting on Multiple Futexes

The original futex() syscall waits on exactly one futex word. The futex2 family generalizes this; the piece merged so far is futex_waitv(), a dedicated syscall (number aside from the multiplexed futex()) that lets a thread wait on an array of up to FUTEX_WAITV_MAX (128) futexes at once and wake on the first of them (futex_waitv(2)). Each element is a struct futex_waitv { __u64 val; __u64 uaddr; __u32 flags; __u32 __reserved; } with per-element FUTEX2_* flags (FUTEX2_SIZE_U32, FUTEX2_PRIVATE, FUTEX2_NUMA). The motivating consumer was game/graphics runtimes — notably Wine/Proton emulating the Windows WaitForMultipleObjects semantics, which the single-futex futex() could not express efficiently. In v6.12 only the 32-bit size is actually implemented (futex_flags_valid() rejects FLAGS_SIZE_64 etc.), even though the ABI reserves bits for 8/16/64-bit futexes. Internally futex_waitv() parses the array into struct futex_vectors, enqueues each on its bucket in a careful two-pass setup (futex_wait_multiple_setup() in waitwake.c) that pins all keys before sleeping to avoid losing a wakeup, then sleeps until any one is woken and returns its index.

Dating the futex2 syscalls by existence-check

The two uncertainty flags that previously stood here have been resolved by fetching kernel/futex/syscalls.c at successive release tags and grepping for each SYSCALL_DEFINE. Fetching the same path at several tags and recording presence/absence dates an addition precisely, which is more reliable than a man page’s HISTORY section (and far more reliable than docs.kernel.org, which always serves the latest documentation regardless of which release you think you are reading about).

futex2 syscall	x86-64 number	Signature at v6.12	First tag where present	Verified by
futex_waitv	449	(struct futex_waitv *waiters, unsigned nr_futexes, unsigned flags, struct __kernel_timespec *timeout, clockid_t clockid)	v5.16	absent from kernel/futex.c at v5.15 (the file was still monolithic), present as SYSCALL_DEFINE5(futex_waitv in kernel/futex/syscalls.c at v5.16
futex_wake	454	(void *uaddr, unsigned long mask, int nr, unsigned flags)	v6.7	SYSCALL_DEFINE4(futex_wake absent at v6.6, present at v6.7
futex_wait	455	(void *uaddr, unsigned long val, unsigned long mask, unsigned flags, struct __kernel_timespec *timeout, clockid_t clockid)	v6.7	SYSCALL_DEFINE6(futex_wait absent at v6.6, present at v6.7
futex_requeue	456	(struct futex_waitv *waiters, unsigned flags, int nr_wake, int nr_requeue)	v6.7	SYSCALL_DEFINE4(futex_requeue absent at v6.6, present at v6.7

(The syscall numbers are read from arch/x86/entry/syscalls/syscall_64.tbl at v6.12; note the gap — 449 was allocated two years before 454–456.)

The second flag asked whether the NUMA and variable-size bits of the futex2 ABI are usable in v6.12 or merely reserved. They are reserved, and the gate is one macro. futex.h at v6.12 defines:

#define FUTEX2_VALID_MASK (FUTEX2_SIZE_MASK | FUTEX2_PRIVATE)

Every futex2 entry point begins if (flags & ~FUTEX2_VALID_MASK) return -EINVAL;, so passing FUTEX2_NUMA (0x04) is rejected before futex2_to_flags() ever gets the chance to turn it into FLAGS_NUMA. Separately, futex_flags_valid() rejects every size except FLAGS_SIZE_32 with the comment “Only 32bit futexes are implemented — for now”, so FUTEX2_SIZE_U8/U16/U64 are equally dead at v6.12.

Existence-checking the same macro forward through the tags shows exactly when that changed:

Tag	FUTEX2_VALID_MASK
v6.12, v6.13, v6.14, v6.15	(FUTEX2_SIZE_MASK | FUTEX2_PRIVATE)
v6.16 and later (checked through v7.1)	(FUTEX2_SIZE_MASK | FUTEX2_NUMA | FUTEX2_MPOL | FUTEX2_PRIVATE)

So NUMA futexes became reachable from userspace in Linux 6.16, together with a new FUTEX2_MPOL flag that does not exist in the v6.12 UAPI header at all. This is directly observable: probing the running kernel (7.1.13) with raw syscalls gives

futex_wake(SIZE_U32|PRIVATE, nr=1)             ret=0    errno=0   ok
futex_wake(SIZE_U32|NUMA)                      ret=0    errno=0   ok
futex_wake(SIZE_U64)  -- 64-bit futex          ret=-1   errno=22  Invalid argument
futex_waitv(1 elem, val matches, 5ms abs timeout) ret=-1 errno=110 Connection timed out
futex_waitv(1 elem, val MISMATCH -> immediate) ret=-1   errno=11  Resource temporarily unavailable
futex_waitv(nr=0)  -- bounds check             ret=-1   errno=22  Invalid argument
futex_waitv(nr=129 > FUTEX_WAITV_MAX)          ret=-1   errno=22  Invalid argument
futex_waitv(flags=1) -- 'no flags for now'     ret=-1   errno=22  Invalid argument

The NUMA flag is accepted on 7.1.13 and would be EINVAL on a 6.12 kernel — a live demonstration of the version boundary. 64-bit futexes are still rejected on 7.1.13, so FUTEX2_SIZE_U64 has been reserved-but-unimplemented for at least the five years since the ABI was designed. The futex_waitv rows confirm the v6.12 source reading directly: ETIMEDOUT when the value matched and the 5 ms absolute deadline expired, EAGAIN when the value did not match (the same verify-then-sleep contract as FUTEX_WAIT), EINVAL for nr_futexes == 0 and for nr_futexes > FUTEX_WAITV_MAX (128), and EINVAL for any non-zero flags — matching the source comment “This syscall supports no flags for now.”

Uncertain

Verify: the mask argument width rule for futex2. Reason: an initial probe passed ~0UL as the mask to futex_wake and got EINVAL, which looked like “the syscall does not exist”; the actual cause is futex_validate_input(), which computes bits = 8 * futex_size(flags) = 32 for a FUTEX2_SIZE_U32 futex and rejects any val or mask with bits set above that. Passing 0xFFFFFFFF succeeds. I read this from the v6.12 source and reproduced the behaviour on 7.1.13, but have not confirmed the rule is documented anywhere user-facing. To resolve: check whether futex_wake(2) / futex_wait(2) man pages exist and state the mask width constraint. uncertain

---

Demonstrated: the Value Check Is What Makes FUTEX_WAIT Safe

The verify-then-sleep contract described above is not a nicety — it is the only thing standing between a correct lock and a permanent hang, and it can be exhibited in about forty lines of C. The experiment uses two threads and a pthread_barrier_t to line them up on each iteration. The waker stores 1 into the word and immediately issues FUTEX_WAKE. The waiter issues FUTEX_WAIT(word, expect) with a 100 ms timeout, and the variable under test is which value it passes as expect:

• Correct: pass 0 — the value whose presence means “the condition has not happened yet, I must sleep.”
• Buggy: re-read the word after the waker has already moved and pass whatever is there now (1).

Three runs on the 7.1.13 host, 32 CPUs, gcc -O2 -pthread:

--- A: tight race, correct expected value ---
N=2000 late=0 stale=0 -> slept(woken)=1 EAGAIN=1999 ETIMEDOUT=0  wall=3ms
--- B: waiter deliberately late, correct expected value ---
N=20 late=1 stale=0 -> slept(woken)=0 EAGAIN=20 ETIMEDOUT=0  wall=21ms
--- C: waiter late, STALE expected value (the classic bug) ---
N=20 late=1 stale=1 -> slept(woken)=0 EAGAIN=0 ETIMEDOUT=20  wall=2023ms

Read those three lines together and the whole mechanism falls out:

• In A, both threads run flat out. In 1,999 of 2,000 iterations the waiter lost the race — the store landed before the kernel could take the bucket lock and re-read the word — and the kernel refused to sleep, returning EAGAIN. Exactly once the waiter got in first and was legitimately woken. Total wall time: 3 ms. Losing the race is the common case and it costs nothing.
• In B, the waiter is forced to lose by sleeping 1 ms after the barrier, so the FUTEX_WAKE has definitely already been issued and returned before FUTEX_WAIT is even entered. Twenty out of twenty EAGAIN, 21 ms wall time — the 20 ms is the deliberate usleep, not blocking. A wakeup that arrived before the wait is not lost, because the value check turns it into a refusal to sleep.
• In C, the timing is identical to B but the waiter passes the post-change value. The kernel now finds uval == val, concludes the condition still holds, and parks the thread. Twenty out of twenty ETIMEDOUT, wall time 2,023 ms ≈ 20 × 100 ms. Every single wait blocked on a condition that was already true. Without the 100 ms timeout this is a permanent hang.

Case C is the lost wakeup, reproduced deterministically. It is worth being precise about where the bug lives: the kernel behaved correctly in all three runs — it slept because it was told the caller still needed to sleep. The defect is a userspace one, and it is why Drepper’s mutex3 calls futex_wait(&val, 2) with the literal constant 2 rather than with a freshly-read value. The expected value must be the one that justified the decision to block, captured at the moment that decision was made, not re-sampled afterwards.

This also explains why EAGAIN from FUTEX_WAIT must never be treated as an error: in run A it was the correct outcome 99.95% of the time.

Failure Modes and Common Misunderstandings

“The futex word is the lock — the kernel reads it.” For plain futexes, false: the value is entirely opaque to the kernel, which only ever compares it to the val the caller passed. For PI and robust futexes it is true — those impose the TID/WAITERS/OWNER_DIED layout precisely because the kernel must interpret ownership. Conflating these is the most common futex misconception.

Lost wakeups from skipping the kernel re-read. A userspace lock implementation that calls FUTEX_WAIT without re-reading and re-checking the word after a failed CAS (or that passes a stale val) can sleep forever: the kernel’s uval == val check is what makes the wait safe, but only if val reflects the value the thread last observed. Drepper’s “Futexes Are Tricky” is the canonical catalogue of these userspace-side bugs (cited in Futex and OS Synchronization Primitives).

Stale EWOULDBLOCK/EAGAIN is normal, not an error. FUTEX_WAIT returning EAGAIN simply means the word changed before the kernel could enqueue the waiter — userspace must loop and retry, never treat it as fatal.

Private vs shared key mismatch. Using FUTEX_WAIT_PRIVATE on one side and a shared wait on the other (or vice versa) yields different keys for the same memory, so waiters and wakers never meet and the wakeup is silently lost. glibc is careful to use the private flag consistently for process-local mutexes.

Robust list corruption only leaks, never misfires. Because handle_futex_death() re-validates the owner TID against the dying task and bounds the walk at ROBUST_LIST_LIMIT, a buggy or malicious robust list cannot cause the kernel to wake the wrong waiter or set OWNER_DIED on a futex the task did not own — the worst outcome is that a genuinely-held lock is not cleaned up.

---

See Also

What this note owns. The vault has five futex notes and they are deliberately non-overlapping. This note owns the in-kernel mechanism: the futex_key/hash-bucket machinery, the complete op and flag tables, the PI-futex word layout and pi_state glue, the robust-list protocol, the futex2 syscall family and its version boundaries, and the measured demonstration that the fast path costs zero syscalls. It does not own: the Go runtime’s futexsleep/futexwakeup (Futex and OS Synchronization Primitives); the cross-process shared-futex story (Futexes Across Process Boundaries); how POSIX mutexes, condvars, rwlocks, semaphores and barriers map onto futex ops (pthread Synchronization and Futexes); or the shared-memory coordination layer (Synchronizing Shared Memory with Futexes and Semaphores). It also does not re-derive the rt_mutex PI chain walk (Priority Inheritance and the RT-Mutex) or the in-kernel struct mutex, spinlock_t and rw_semaphore — those are kernel-internal locks that protect kernel data and never touch user memory (Kernel Mutexes, Kernel Spinlocks, Read-Write Semaphores).

• Futex and OS Synchronization Primitives — the generic/Go-runtime view: how the Go runtime parks OS threads (futexsleep/futexwakeup) on this exact kernel mechanism; the userspace fast-path/lost-wakeup discussion (Drepper)
• Priority Inheritance and the RT-Mutex — the rt_mutex and PI chain walk that PI-futexes are built on; this note covers only the futex-word glue
• Kernel Mutexes — the in-kernel struct mutex (a different thing from a userspace futex-backed mutex; both park tasks but the kernel mutex protects kernel data)
• Memory Barriers in the Linux Kernel and The Linux Kernel Memory Model — the barriers (A)/(B) that make the lock-free wakeup-skip safe
• lockdep Runtime Lock Validator — the deadlock detector that double_lock_hb()’s address-ordered locking is written to satisfy
• Futexes Across Process Boundaries — the same word in MAP_SHARED memory: why the shared key path pins a page, and what PTHREAD_PROCESS_SHARED costs
• pthread Synchronization and Futexes — which futex op each POSIX primitive actually issues
• Synchronizing Shared Memory with Futexes and Semaphores — the coordination layer over mmaped regions
• Acquire Release and Fence Semantics — the C11 ordering vocabulary the userspace half of a futex protocol is written in
• ptmalloc and glibc malloc — a heavy in-process user of glibc’s lll_lock, i.e. of exactly this mechanism
• sigaction and Signal Handlers — why FUTEX_WAIT can return EINTR, and why glibc’s __futex_abstimed_wait_cancelable64 exists
• Linux IPC MOC — home of the userspace-facing futex() syscall API as an inter-process synchronization surface
• Linux Kernel Synchronization MOC — parent MOC (section F, Correctness)