vault

The cgroup pids Controller

15 min read

The cgroup pids Controller

The pids controller (the kernel calls it the process number controller) is the cgroups-v2 facility that bounds how many tasks a cgroup subtree may contain. It exposes essentially two knobs — pids.max, the hard limit, and pids.current, the live count — and enforces the limit at exactly one point: every fork()/clone() charges the new task to the cgroup, and a fork that would push the count over the limit fails with -EAGAIN rather than creating the task (per the v6.12 cgroup-v2.rst “PID” section and kernel/cgroup/pids.c). It is the simplest controller in the system — a single hierarchically-summed counter compared against a limit — and it exists independently of the memory controller for one decisive reason: PIDs are a globally finite resource. The kernel can allocate only up to pid_max process IDs system-wide, and a fork bomb exhausts that pool long before it exhausts memory, so a controller that limits memory cannot protect against task-table exhaustion. As pids.c’s own header puts it, “Since it is trivial to hit the task limit without hitting any kmemcg limits in place, PIDs are a fundamental resource.”

This note is pinned to Linux 6.12 LTS (released 2024-11-17), with the implementation read directly from kernel/cgroup/pids.c in that tree. The controller’s behaviour and interface are stable across 6.12 and 6.18 LTS; the pids.peak file and the pids_localevents boot parameter discussed below are present in 6.12.

Mental Model — A Counter and a Limit at the Fork Boundary

The right way to think about pids is almost embarrassingly simple after the memory or io controllers: it is a counter (pids.current) and a limit (pids.max), and the only enforcement event is task creation. There is no continuous throttling, no proportional sharing, no QoS — a fork either succeeds (and increments the counter up the hierarchy) or it is refused with -EAGAIN. Everything else follows from that.

flowchart TB
  FORK["Process calls fork() / clone()<br/>(copy_process in the kernel)"]
  FORK --> CANFORK["pids_can_fork()<br/>→ pids_try_charge(+1)"]
  CANFORK --> WALK["Walk UP the hierarchy:<br/>for each ancestor cgroup,<br/>atomic +1 then compare to its limit"]
  WALK --> OK{"new <= limit<br/>at every level?"}
  OK -->|yes| SUCCEED["Charge stands.<br/>pids.current incremented<br/>at every ancestor. Task created."]
  OK -->|no| REVERT["Revert every +1 already done,<br/>record the over-limit cgroup,<br/>fire pids.events"]
  REVERT --> FAIL["fork() returns -EAGAIN.<br/>No task created."]
  EXIT["Task exits (pids_release)"] --> UNCHARGE["atomic -1 at every ancestor"]

The entire pids controller in one diagram. What it shows: the only place the limit matters is fork/clone; the controller atomically charges +1 to the forking cgroup and every ancestor, and if any level would exceed its limit it rolls back all the increments and fails the fork with -EAGAIN. The insight to take: because the charge walks up the tree, a parent’s pids.current is the sum of its whole subtree, and the most restrictive limit anywhere on the path wins — exactly the hierarchical containment a fork bomb needs.

What It Accounts — Tasks, Charged Up the Hierarchy

The unit the controller counts is a task — and the doc is precise that “PIDs used in this controller refer to TIDs, process IDs as used by the kernel” (cgroup-v2.rst, “PID”). A TID (thread ID) is the kernel’s per-schedulable-entity identifier; what userspace calls a “process” with multiple threads is, to the kernel, several tasks sharing a thread-group ID. So a process that spawns ten threads consumes eleven against pids.max, not one. This matters: thread-heavy applications (a JVM, a Go service with many OS threads) can hit pids.max even with a single “process.”

pids.current is “the number of processes currently in the cgroup and its descendants” (cgroup-v2.rst). This is the hierarchical-summation property, and it is implemented directly in the charge path. In pids.c, pids_charge() and pids_try_charge() both loop for (p = pids; parent_pids(p); p = parent_pids(p)) — walking from the leaf cgroup up to (but not including) the root — and do atomic64_add_return(num, &p->counter) at each level (pids.c, lines 145 and 166). So creating one task increments the counter of the cgroup and every ancestor. The header comment states the resulting invariant: “pids.current tracks all child cgroup hierarchies, so parent/pids.current is a superset of parent/child/pids.current.”

The Enforcement Path — pids_can_fork, pids_try_charge, and -EAGAIN

The controller hooks the kernel’s fork machinery through the can_fork callback. When a process forks, copy_process() calls each subsystem’s can_fork; the pids controller’s is pids_can_fork(), which simply calls pids_try_charge(pids, 1, &pids_over_limit) (pids.c, line 273). pids_try_charge is where the limit lives, and its logic is worth walking line by line because it is the whole controller:

static int pids_try_charge(struct pids_cgroup *pids, int num, struct pids_cgroup **fail)
{
    struct pids_cgroup *p, *q;
 
    for (p = pids; parent_pids(p); p = parent_pids(p)) {
        int64_t new = atomic64_add_return(num, &p->counter);   // optimistically charge
        int64_t limit = atomic64_read(&p->limit);
 
        if (new > limit) {            // this ancestor would be over its limit
            *fail = p;                // remember which one, for events
            goto revert;
        }
        pids_update_watermark(p, new);
    }
    return 0;                          // charged at every level, success
 
revert:
    for (q = pids; q != p; q = parent_pids(q))
        pids_cancel(q, num);           // undo the +1 at every level below the failure
    pids_cancel(p, num);               // and at the failing level itself
    return -EAGAIN;
}

The design is optimistic: it increments each ancestor’s counter first (atomic64_add_return), then checks against that level’s limit. If any level’s new count exceeds its limit, it records the offending cgroup in *fail, jumps to revert, and rolls back every increment it already made — both the levels below the failure (the for q != p loop) and the failing level itself (pids_cancel(p, num)). It then returns -EAGAIN. This optimistic-charge-then-revert avoids holding a lock across the whole hierarchy walk; the counters are atomic64_t, so concurrent forks on different CPUs are correct without a mutex.

Back in pids_can_fork, if pids_try_charge returned an error, it calls pids_event() to record the failure, then returns the error code up to copy_process(), which aborts the fork and returns -EAGAIN to userspace (pids.c, line 279). The doc states the user-visible contract plainly: “it is not possible to violate a cgroup PID policy through fork() or clone(). These will return -EAGAIN if the creation of a new process would cause a cgroup policy to be violated” (cgroup-v2.rst). To userspace this looks like the system call temporarily failing — fork(2) documents EAGAIN as “a system-imposed limit on the number of threads was encountered,” which is exactly what a pids.max ceiling is.

The symmetric un-charge happens when a task exits: the release callback pids_release() calls pids_uncharge(pids, 1), which walks the same hierarchy doing atomic64_add_negative(-num, ...) at each level (pids.c, lines 294 and 128). And if a fork is charged but then aborted for an unrelated reason after can_fork succeeded, pids_cancel_fork() un-charges the speculative +1 — so the counter never leaks.

The Interface Files

The controller registers a small set of files on every non-root cgroup (all flagged CFTYPE_NOT_ON_ROOT in pids_files[] — the root cannot have a PID limit, “for obvious reasons,” since limiting the root would limit the whole machine) (pids.c, line 390):

  • pids.max — read-write single value; the default is max (unlimited). Writing a number sets the hard limit; writing the literal string max removes it. The write handler pids_max_write rejects values < 0 or >= PIDS_MAX with -EINVAL (pids.c, line 301).
  • pids.current — read-only single value; the current task count for the cgroup and its descendants. Implemented as a bare atomic64_read(&pids->counter).
  • pids.peak — read-only; “the maximum value that the number of processes … has ever reached” — the high-water mark, maintained by pids_update_watermark during charging (cgroup-v2.rst).
  • pids.events — read-only flat-keyed; its max field counts “the number of times the cgroup’s total number of processes hit the pids.max limit.” A change here generates a file-modified notification, so a userspace monitor can poll() it and react to throttling.
  • pids.events.local — like pids.events but counting only events local to this cgroup (non-hierarchical).

A worked example — limit a cgroup to 100 tasks and observe enforcement:

# Limit the cgroup to 100 tasks
echo 100 > /sys/fs/cgroup/mygroup/pids.max
 
# Move the current shell into it (writing a PID to cgroup.procs)
echo $$ > /sys/fs/cgroup/mygroup/cgroup.procs
 
cat /sys/fs/cgroup/mygroup/pids.current   # e.g. 1 (just this shell)
 
# A fork bomb inside this cgroup now hits the wall at 100:
#   :(){ :|:& };:
# every fork past 100 returns EAGAIN; the bomb cannot exhaust the host's PIDs.
 
cat /sys/fs/cgroup/mygroup/pids.events    # max <N>  — N = times the limit was hit

Why It Exists Independently of Memory — PIDs Are Globally Finite

The deepest point about this controller is why it is a separate controller at all. Naively, one might think a memory limit suffices: a runaway process tree consumes memory per task, so memory.max should stop it. It does not — and the reason is that the system-wide PID space is a hard, separate ceiling that is exhausted before memory.

The kernel allocates process IDs from a finite pool bounded by pid_max, whose compile-time default and absolute limit are defined in include/linux/threads.h. On a standard 64-bit build, PID_MAX_DEFAULT is 0x8000 = 32768 (IS_ENABLED(CONFIG_BASE_SMALL) ? 0x1000 : 0x8000), and the absolute ceiling PID_MAX_LIMIT is 4 × 1024 × 1024 = 4,194,304 (“A maximum of 4 million PIDs should be enough for a while”) (threads.h, lines 28 and 34). The runtime pid_max (tunable via /proc/sys/kernel/pid_max) sits between these. The controller’s own PIDS_MAX sentinel — the value that means “no limit” — is PID_MAX_LIMIT + 1ULL, deliberately one beyond the largest real allocation so the over-limit test “can never fail” when the limit is max (pids.c, line 38 and the comment at line 174).

Now consider a fork bomb — a process that does nothing but fork copies of itself as fast as possible (:(){ :|:& };:). Each task is cheap in memory (a fresh fork shares pages copy-on-write and may touch only kilobytes before forking again), but each consumes one slot in the global PID space and one entry in the kernel’s task table. The pool drains in a fraction of a second. Once it is empty, no process anywhere on the system can fork — including the administrator’s shell, login, or any recovery tool — which is precisely the denial-of-service a fork bomb achieves. The pids.c header names this exactly: “a fork bomb is likely to exhaust the number of tasks before hitting memory restrictions” (pids.c / cgroup-v2.rst). The doc reinforces it: “The number of tasks in a cgroup can be exhausted in ways which other controllers cannot prevent, thus warranting its own controller.”

So pids is not redundant with memory; it guards an orthogonal finite resource. A container with a generous memory.max but no pids.max can still take down the host with a fork bomb. This is why every serious container runtime sets a PID limit by default.

Migration — Why pids.current Can Exceed pids.max

Moving an existing task between cgroups goes through a different charge path than forking, and that difference is the whole reason the limit is not an instantaneous invariant. When cgroup core migrates tasks, it calls the can_attach callback — here pids_can_attach() — which, for each task being moved, charges the destination cgroup and un-charges the source (pids.c, line 200):

cgroup_taskset_for_each(task, dst_css, tset) {
    ...
    pids_charge(pids, 1);          // destination: UNCONDITIONAL charge
    pids_uncharge(old_pids, 1);    // source: give the slot back
}
return 0;                           // attach NEVER fails on the pids limit

The critical detail is that migration calls pids_charge() — not pids_try_charge(). pids_charge is documented in its header as the function that “does not follow the pid limit set. It cannot fail and the new pid count may exceed the limit” (pids.c, line 145). So pids_can_attach unconditionally returns 0 (success). This is a deliberate design choice: cgroup core treats organisational operations (moving tasks, reorganising the tree) as administrative actions that must not be blocked by resource policies — only resource-creating operations (fork) are gated. pids_cancel_attach mirrors the charge to roll it back if some other subsystem vetoes the migration.

The consequence, stated plainly in the doc: “Organisational operations are not blocked by cgroup policies, so it is possible to have pids.current > pids.max. This can be done by either setting the limit to be smaller than pids.current, or attaching enough processes to the cgroup such that pids.current is larger than pids.max” (cgroup-v2.rst). So the limit is a ceiling on growth by forking, not a hard invariant on the count. After such an overshoot, the count is “stuck” above the limit until tasks exit, and no new fork succeeds until it falls back below.

Failure Modes and Subtleties

pids.current can legitimately exceed pids.max — as the migration section above shows, via either shrinking the limit under the live count or attaching processes past it. This is by design, not a bug.

It counts threads, not just processes. As noted, a TID-counting controller charges every thread. A multithreaded service can hit pids.max with what an operator thinks of as “one process,” producing baffling EAGAIN/Resource temporarily unavailable errors that look like a kernel bug but are a too-low PID limit. The diagnosis is cat pids.current vs pids.max and remembering threads count.

-EAGAIN is overloaded. fork returns EAGAIN both for hitting pids.max and for hitting the system-wide RLIMIT_NPROC / pid_max. Distinguishing requires checking the cgroup’s pids.current/pids.max and pids.events (whose max counter increments precisely when this cgroup’s limit caused the rejection). A bumped pids.events:max is the fingerprint that the cgroup limit, not a global limit, was the cause.

pids.peak is deliberately approximate. The high-water mark behind pids.peak is updated by pids_update_watermark(), which the source openly comments is “racy, but we don’t need perfectly accurate tallying of the watermark, and this lets us avoid extra atomic overhead” — it does a plain READ_ONCE/WRITE_ONCE rather than an atomic compare-and-swap (pids.c, line 96). So under heavy concurrent forking the recorded peak can lag the true momentary maximum slightly. This is a fine trade: pids.peak is a capacity-planning hint (“how close did we get to the limit?”), not an enforcement input, so a few-task race is irrelevant while the saved atomic op on the hot fork path is not.

Hierarchical vs local event accounting. Whether pids.events:max counts only this cgroup’s own rejections or any rejection in its subtree depends on the pids_localevents boot parameter. By default (without it), pids.events.max “represents any pids.max enforcement across cgroup’s subtree”; with pids_localevents set, it “restores v1-like behavior … only local … fork failures are counted” (cgroup-v2.rst, boot-parameters section). This is implemented in pids_event() and __pids_events_show() via the CGRP_ROOT_PIDS_LOCAL_EVENTS flag (pids.c, lines 254 and 367). pids.events.local always gives the non-hierarchical view regardless of the flag.

Alternatives and When to Choose Them

There is no within-controller alternative — pids has exactly one policy (a hard limit) — so the comparison is against other ways to bound task counts. The classic predecessor is RLIMIT_NPROC (ulimit -u), the per-user process limit enforced via setrlimit. RLIMIT_NPROC is keyed by real UID, so it is escaped trivially by any workload that can change UID, and it does not compose hierarchically — two containers run by the same user share one limit. The pids controller is keyed by cgroup membership instead, which is exactly the boundary a container is, making it the correct tool for per-container limits. The two are complementary: RLIMIT_NPROC as a coarse per-user backstop, pids.max as the precise per-container ceiling.

Within cgroups, pids is the lightest controller — pure atomic counters, no per-bio or per-page bookkeeping — which is part of why it is enabled essentially universally. There is rarely a reason not to set a PID limit on an untrusted workload.

Production Notes

The controller was written by Aleksa Sarai in 2015 (copyright header, pids.c), and exists precisely to make per-container fork-bomb protection possible. In practice, container runtimes and orchestrators wire workload PID limits onto pids.max for the workload’s cgroup. Kubernetes does this through the kubelet: per-Pod PID limiting is configured at the node level — the same per-Pod budget applied to every Pod on that node — via the kubelet’s --pod-max-pids command-line parameter or the PodPidsLimit field in the kubelet configuration file; separately, PIDs can be reserved for node/system overhead via the pid=<number> argument to --system-reserved/--kube-reserved (Kubernetes “Process ID Limits And Reservations”). The Kubernetes docs frame the goal exactly as this controller’s: per-Pod limiting “protect[s] one Pod from another” but does not by itself stop all Pods collectively from exhausting the node — which is why the node reservation exists alongside it. The recommended posture is to always set a finite pids.max for any workload you do not fully trust, because the cost of doing so is negligible and the cost of not doing so is a host-wide PID exhaustion that takes down every workload on the machine, not just the offender.

Uncertain

Verify: that Docker exposes a per-container PID limit as docker run --pids-limit mapping onto pids.max. Reason: this is a memory-based claim — the Docker CLI reference was not successfully fetched this session. To resolve: confirm against the current docker run / docker container run CLI reference and the runtime’s cgroup-writing code. The Kubernetes mapping above is pinned to the fetched Kubernetes docs. uncertain

See Also

Graph View

Backlinks