vault

ipcs ipcrm and System V IPC Limits

15 min read

ipcs ipcrm and System V IPC Limits

Because System V IPC objects persist in the kernel until something explicitly removes them, every Linux system needs a way to see what is resident and to delete leaked objects, and every kernel imposes limits on how many such objects (and how large) may exist. The two userspace commands are ipcs (list message queues, semaphore sets, and shared-memory segments, with their keys, ids, owners, permissions, and attach counts) and ipcrm (remove an object by its id or its key). The limits are a handful of sysctls under /proc/sys/kernel/ — shmmax/shmall/shmmni for shared memory, msgmax/msgmnb/msgmni for message queues, and the four-valued sem for semaphores. The most important, frequently-misunderstood fact about the modern defaults is that the shared-memory size limits are effectively disabled: at Linux 6.12, SHMMAX and SHMALL both default to ULONG_MAX − 2²⁴, i.e. ~18 exabytes, so the historical “raise SHMMAX before installing your database” ritual is obsolete on current kernels (include/uapi/linux/shm.h, v6.12; ipc/shm.c, v6.12).

This is the operational/administrative companion to System V IPC Overview and Keys. It walks ipcs and ipcrm first, then dissects each kernel limit, its sysctl, its verified v6.12 default, and its per-namespace scoping.

Mental Model

ipcs and ipcrm are to System V IPC what ls and rm are to files — except the “directory” is a kernel namespace, not a path. Both commands read their listing from the /proc/sysvipc/ pseudo-files (/proc/sysvipc/shm, .../msg, .../sem) when /proc is mounted, falling back to the *ctl(IPC_STAT) syscalls otherwise (ipcs(1)). The limits are ceilings the kernel checks inside the *get() and operation syscalls; exceeding one yields ENOSPC, EINVAL, or E2BIG rather than a silent truncation.

flowchart TB
  subgraph CMD["Admin commands"]
    IPCS["ipcs -a / -m / -q / -s / -l / -u"]
    IPCRM["ipcrm -m id / -M key / -a"]
  end
  subgraph PROC["/proc — read & tune"]
    SYSVIPC["/proc/sysvipc/{shm,msg,sem}<br/>(live object listing)"]
    SYSCTL["/proc/sys/kernel/{shmmax,shmall,shmmni,<br/>msgmax,msgmnb,msgmni,sem}<br/>(per-namespace limits)"]
  end
  subgraph K["Kernel — per IPC namespace"]
    NS["ipc_namespace fields:<br/>shm_ctlmax, shm_ctlall, shm_ctlmni<br/>msg_ctlmax, msg_ctlmnb, msg_ctlmni<br/>sem_ctls[4]"]
  end
  IPCS -- "reads" --> SYSVIPC
  IPCS -- "-l reads" --> SYSCTL
  IPCRM -- "IPC_RMID syscall" --> K
  SYSCTL <-- "proc_handlers" --> NS
  K --> SYSVIPC

The admin-facing surface of System V IPC. What it shows: ipcs reads the live object list from /proc/sysvipc/ and the ceilings from /proc/sys/kernel/; ipcrm issues IPC_RMID; the sysctls map to per-namespace fields inside struct ipc_namespace. The insight: there is no global System V state — every limit and every object lives in one IPC namespace, so a container with its own namespace has its own independent limits and its own leaked objects.

ipcs — Seeing What Is Resident

Run with no arguments, ipcs “displays information about System V inter-process communication facilities” for all three resource types at once (ipcs(1)). The resource-selection options narrow it to one type — -m/--shmems (shared memory), -q/--queues (message queues), -s/--semaphores (semaphore arrays), -a/--all (all three, the default). The output-format options, of which only the last given takes effect, change what columns you see:

  • -c/--creator — show the creator and owner UIDs/GIDs (the cuid/cgid vs uid/gid distinction from the [[System V IPC Overview and Keys|ipc_perm header]]).
  • -p/--pid — “Show PIDs of creator and last operator” (e.g. for shm, the PID that created it and the PID of the last shmat/shmdt).
  • -t/--time — show last-operation timestamps (shm_atime, shm_dtime, msg_stime, etc.).
  • -u/--summary — a status summary (totals: how many segments, how much memory in use) rather than a per-object listing.
  • -l/--limits — print the limits (the sysctl ceilings) for the selected type rather than the live objects.
  • -i ID/--id ID — “Show full details on just the one resource element identified by id”; must be combined with -m, -q, or -s.

A default ipcs -m (shared memory) prints, per segment: the key (the ftok/hard-coded key, shown as hex, or 0x00000000 for an IPC_PRIVATE object that has no key), the shmid (the id you pass to ipcrm), the owner, the perms (the octal mode from shm_perm.mode), the segment bytes, the nattch (number of processes currently attached via shmat), and the status — where dest means the segment has been marked for destruction (SHM_DEST, an IPC_RMID was issued but processes are still attached) and locked means it is mlock-pinned. A representative listing:

$ ipcs -m

------ Shared Memory Segments --------
key        shmid      owner   perms   bytes      nattch  status
0x00000000 32768      postgres 600    4194304    3       dest
0x52010d2f 65537      app      660    1048576    1

The first row is a dest-flagged segment: key 0x0 (it was IPC_PRIVATE), three processes still attached, already marked for removal — it will vanish when nattch reaches 0. Reading this correctly is the difference between “this is leaking” and “this is mid-teardown.” ipcs -m -i 32768 would dump full details on just that one segment, including the creator/last-operator PIDs and the attach/detach times.

ipcrm — Removing Leaked Objects

ipcrm “removes System V inter-process communication (IPC) objects” — and you must be the object’s creator, its owner, or the superuser to do so (ipcrm(1)). Its defining design choice is lowercase = by id, uppercase = by key:

  • Message queues: -q/--queue-id msgid removes by id; -Q/--queue-key msgkey removes by key.
  • Semaphores: -s/--semaphore-id semid by id; -S/--semaphore-key semkey by key.
  • Shared memory: -m/--shmem-id shmid by id; -M/--shmem-key shmkey by key.

Numbers may be decimal, hex (0x…), or octal (leading 0) — convenient because ipcs prints keys in hex. There are also POSIX variants (--posix-shmem, --posix-mqueue, --posix-semaphore) for POSIX named objects, and -a/--all to “remove all resources” (optionally limited to specified types) — which the man page warns against using casually, since “it is possible that other programs depend on these resources” at startup. A typical leaked-object cleanup:

ipcs -m                       # find the leaked shmid, say 32768
ipcrm -m 32768                # remove by id
# or, if you only know the key the program used:
ipcrm -M 0x52010d2f           # remove by key (uppercase M)

The crucial behavioural asymmetry: shared memory removal is deferred, semaphore and message-queue removal is immediate. Per the man page, “A shared memory object is only removed after all currently attached processes have detached” it (ipcrm(1)) — ipcrm -m on an attached segment merely sets the SHM_DEST flag (the dest status above), and the segment is freed only when the last shmdt happens. Message queues and semaphores have no such grace: IPC_RMID destroys them instantly, immediately waking any process blocked in msgrcv/semop with EIDRM. This mirrors the kernel logic in ipc/shm.c, where if shp->shm_nattch is non-zero on IPC_RMID the segment “receives SHM_DEST marking and becomes inaccessible; otherwise shm_destroy() executes immediately” (ipc/shm.c, v6.12).

The Kernel Limits and Their Sysctls

Each limit is a field of the per-namespace struct ipc_namespace, exposed as a writable sysctl under /proc/sys/kernel/. The registrations live in ipc/ipc_sysctl.c; all of the substantive ones operate on ns->… fields, confirming they are per-IPC-namespace, not global (ipc/ipc_sysctl.c, v6.12). The defaults are assigned in the three *_init_ns() functions and ultimately come from the UAPI header constants.

Shared memory — shmmax, shmall, shmmni

These map to ns->shm_ctlmax, ns->shm_ctlall, ns->shm_ctlmni, initialised in shm_init_ns() (ipc/shm.c, v6.12):

void shm_init_ns(struct ipc_namespace *ns)
{
	ns->shm_ctlmax = SHMMAX;   /* per-segment max size, bytes  */
	ns->shm_ctlall = SHMALL;   /* system-wide total, in pages  */
	ns->shm_ctlmni = SHMMNI;   /* max number of segments       */
	ns->shm_rmid_forced = 0;
	ns->shm_tot = 0;
	ipc_init_ids(&shm_ids(ns));
}

The constants resolve as follows in the v6.12 UAPI header (include/uapi/linux/shm.h, v6.12):

#define SHMMIN 1                     /* min shared seg size (bytes) */
#define SHMMNI 4096                  /* max num of segs system wide */
#define SHMMAX (ULONG_MAX - (1UL << 24))
#define SHMALL (ULONG_MAX - (1UL << 24))
#define SHMSEG SHMMNI                /* max shared segs per process */
  • shmmax — the maximum size of a single segment, in bytes. Default ULONG_MAX − 2²⁴ ≈ 1.8 × 10¹⁹ bytes (~18 EB) on a 64-bit kernel — i.e. effectively unlimited. shmget returns EINVAL if size exceeds it (shmget(2)).
  • shmall — the system-wide ceiling on total shared memory, counted in pages (not bytes). Also ULONG_MAX − 2²⁴ pages by default — likewise effectively unlimited. Exceeding it yields ENOSPC.
  • shmmni — the maximum number of segments in the namespace. Default 4096. Exhaustion yields ENOSPC.

Uncertain

The headline claim — SHMMAX = SHMALL = ULONG_MAX − (1 << 24) at Linux 6.12 — is verified directly against the v6.12 UAPI header and shm_init_ns() (both fetched above), so it is not in doubt. What is worth a flag: the man page shmget(2) attributes this default to “Linux 3.16+”, whereas the LWN discussion of Manfred Spraul’s patch I read does not state the exact landing release. Verify: the first release in which the ULONG_MAX − 2²⁴ default shipped. Reason: man-page says 3.16; LWN thread (April 2014) does not pin the merge. To resolve: git log --oneline -- include/uapi/linux/shm.h around the 3.16 merge window. This does not affect the 6.12 fact. uncertain

The deliberately-not-quite-ULONG_MAX value has a real rationale. Manfred Spraul chose ULONG_MAX − (1 << 24) instead of literal ULONG_MAX specifically to leave headroom for a known userspace anti-pattern: “there are known cases where an application simply tries to increment the value SHMMAX rather than setting it, which causes an overflow” (LWN, Changing the default shared memory limits). Subtracting 2²⁴ (16 MiB) gives such code room to add to the value without wrapping around to a tiny number. The change was motivated by the fact that the old 32 MB SHMMAX had become a perennial install-time obstacle — “it has been routine procedure for several years for users to increase SHMMAX on production systems” (LWN) — so the maintainers simply set it out of the way. The old SHMALL of roughly 8 GB (in the 2.4–3.15 era, per shmget(2)) was raised at the same time.

There is also shm_rmid_forced (ns->shm_rmid_forced, default 0). When set to 1, “all System V shared memory segments will be marked for destruction as soon as the number of attached processes falls to zero” (proc_sys_kernel(5)) — effectively turning off the kernel-persistent lifetime for shm and making segments behave more like POSIX objects. Its proc handler, proc_ipc_dointvec_minmax_orphans, immediately runs shm_destroy_orphaned(ns) on write so existing orphans are reaped at once (ipc/ipc_sysctl.c, v6.12).

Message queues — msgmax, msgmnb, msgmni

These map to ns->msg_ctlmax, ns->msg_ctlmnb, ns->msg_ctlmni, initialised in msg_init_ns() to fixed constants — not the historical memory-scaled value (ipc/msg.c, v6.12):

ns->msg_ctlmax = MSGMAX;   /* 8192  — max bytes in one message      */
ns->msg_ctlmnb = MSGMNB;   /* 16384 — default max bytes in a queue  */
ns->msg_ctlmni = MSGMNI;   /* 32000 — max number of queues          */

with the constants from the header (include/uapi/linux/msg.h, v6.12):

#define MSGMNI 32000   /* <= IPCMNI */   /* max # of msg queue identifiers */
#define MSGMAX  8192   /* <= INT_MAX */  /* max size of message (bytes) */
#define MSGMNB 16384   /* <= INT_MAX */  /* default max size of a message queue */
  • msgmax — the maximum number of bytes in a single message body. Default 8192. msgsnd of a larger message fails with EINVAL.
  • msgmnb — the value used to initialise msg_qbytes for newly created queues, i.e. the default per-queue byte capacity. Default 16384. A privileged process can raise an individual queue’s msg_qbytes beyond this via IPC_SET.
  • msgmni — the maximum number of message-queue identifiers in the namespace. Default 32000.

The msgmni default deserves emphasis because the documentation landscape is full of stale claims that it is auto-computed from RAM. That was true historically: from Linux 2.6.27 to 3.18, the auto_msgmni sysctl recomputed msgmni on memory hotplug or IPC-namespace creation. Since Linux 3.19 this no longer happens — msgmni simply defaults “near the maximum value possible” (32000), and “the content of [auto_msgmni] has no effect … and reads from this file always return the value 0” (proc_sys_kernel(5)). Correspondingly, ipc/ipc_sysctl.c still registers auto_msgmni but only via the read-only-ish handler proc_ipc_auto_msgmni, whose comment notes “writing to auto_msgmni has no effect” (ipc/ipc_sysctl.c, v6.12). Treat any guide that tells you to tune auto_msgmni as out of date.

Semaphores — the four-valued sem

Unlike the others, the semaphore limits are packed into a single sysctl that holds four numbers, mapping to ns->sem_ctls[4]. Reading /proc/sys/kernel/sem yields four whitespace-separated values, “in order: SEMMSL, SEMMNS, SEMOPM, SEMMNI” (proc_sys_kernel(5)). The kernel’s accessor macros confirm the index order (ipc/sem.c, v6.12):

#define sc_semmsl  sem_ctls[0]   /* SEMMSL — max semaphores per set      */
#define sc_semmns  sem_ctls[1]   /* SEMMNS — max semaphores system-wide  */
#define sc_semopm  sem_ctls[2]   /* SEMOPM — max ops per semop() call    */
#define sc_semmni  sem_ctls[3]   /* SEMMNI — max number of sets          */

and sem_init_ns() seeds them from the header constants:

void sem_init_ns(struct ipc_namespace *ns)
{
	ns->sc_semmsl = SEMMSL;   /* 32000 */
	ns->sc_semmns = SEMMNS;   /* SEMMNI * SEMMSL = 32000 * 32000 */
	ns->sc_semopm = SEMOPM;   /* 500   */
	ns->sc_semmni = SEMMNI;   /* 32000 */
	ns->used_sems = 0;
	ipc_init_ids(&ns->ids[IPC_SEM_IDS]);
}

with (include/uapi/linux/sem.h, v6.12):

#define SEMMNI  32000            /* max # of semaphore identifiers */
#define SEMMSL  32000            /* max num of semaphores per id   */
#define SEMMNS  (SEMMNI*SEMMSL)  /* max # of semaphores in system  */
#define SEMOPM  500              /* max num of ops per semop call  */
#define SEMVMX  32767            /* semaphore maximum value        */
  • SEMMSL (index 0) — maximum number of semaphores per set; bounds the nsems argument to semget. Default 32000 (since Linux 3.19, per semget(2)). semget with a larger nsems returns EINVAL.
  • SEMMNS (index 1) — system-wide total number of semaphores across all sets. Default SEMMNI * SEMMSL = 32000 × 32000 = 1.024 × 10⁹. Hitting it yields ENOSPC.
  • SEMOPM (index 2) — maximum operations in a single semop/semtimedop call (System V semaphores uniquely allow an atomic batch of operations). Default 500. A larger batch returns E2BIG.
  • SEMMNI (index 3) — maximum number of semaphore sets (identifiers) in the namespace. Default 32000. Exhaustion yields ENOSPC.

(A separate, non-tunable constant SEMVMX = 32767 caps the value any individual semaphore may hold.) The sem sysctl’s proc handler is special — proc_ipc_sem_dointvec validates the proposed quad with sem_check_semmni() and reverts the whole write on error, so you cannot set an internally inconsistent set of four values (ipc/ipc_sysctl.c, v6.12).

Per-Namespace Scoping — Why Containers Have Their Own Limits

Every limit above is a field of struct ipc_namespace, and every sysctl handler in ipc/ipc_sysctl.c reads or writes ns->… (ipc/ipc_sysctl.c, v6.12). The “namespace” a process sees is determined by its membership in an IPC namespace (created with unshare(CLONE_NEWIPC) or clone(CLONE_NEWIPC)). The consequence is that /proc/sys/kernel/shmmax (and the others) shows and tunes the limits for the calling process’s IPC namespace, not a machine-global value. A container with its own IPC namespace starts from a fresh copy of the defaults and can be tuned independently; a process sharing the host namespace tunes the host’s. Likewise the objects counted against shmmni/msgmni/semmni are only those in the same namespace — which is why ipcs inside a container shows only that container’s objects and why destroying an IPC namespace (last member exits) reaps all its objects at once, a clean leak boundary the host namespace does not enjoy.

Failure Modes and Diagnostics

“shmget fails with ENOSPC even though I have free RAM.” You hit shmmni (4096 segments) or, far less likely now, shmall. Because shmmax/shmall are effectively infinite at v6.12, the segment count limit is the one you actually reach — check ipcs -m | wc -l against shmmni. A flood of leaked segments from crashed workers is the usual cause; ipcrm them and fix the leak.

“semop fails with E2BIG.” Your batch exceeds SEMOPM (500). Split the operation array.

“My old runbook says to set kernel.shmmax in /etc/sysctl.conf before installing Oracle/Postgres.” On 6.12 this is harmless but pointless for the size limits — the defaults are already astronomically high. The databases’ own documentation and installers may still set it for compatibility with ancient kernels. The one shm sysctl still worth touching is sometimes shmmni (raise the segment count) or shm_rmid_forced=1 (auto-reap orphans) — not shmmax.

“ipcs shows a dest segment that won’t go away.” That is a segment already marked for destruction (someone ran ipcrm -m or IPC_RMID) but with nattch > 0. It will disappear when the last attached process detaches or exits. Find the holders via the creator/last-operator PID (ipcs -m -p) and the system-wide picture, then kill or fix them.

“I ipcrm’d a message queue and a blocked reader died.” Expected: IPC_RMID on a queue or semaphore is immediate and wakes blocked msgrcv/semop callers with EIDRM (“identifier removed”). Only shared memory defers. Code that uses System V queues/semaphores must handle EIDRM.

Alternatives and When to Choose Them

For monitoring, ipcs is fine but coarse; production systems usually scrape /proc/sysvipc/{shm,msg,sem} directly or use the lsipc utility (from util-linux, the same package as ipcs/ipcrm), which offers a more parseable, column-selectable output. For cleanup, prefer designing the leak away — use the [[System V Shared Memory|lazy SHM_DEST removal]] (mark for destruction immediately after shmat so it self-cleans), set shm_rmid_forced=1 namespace-wide, or move to POSIX named objects which die on last-close-after-unlink. For limits, the modern stance is to leave the shm size sysctls at their effectively-infinite defaults and only manage the count limits and namespace scoping.

Production Notes

The single biggest real-world takeaway is that the shm-size tuning ritual is dead on current kernels. Years of database install guides, Stack Overflow answers, and vendor docs tell administrators to bump kernel.shmmax/kernel.shmall — advice that was essential on 2.4–3.15 kernels (32 MB SHMMAX, ~8 GB SHMALL) but is inert on anything from Linux 3.16 onward, including 6.12 (LWN; shmget(2)). The limits that still bite are counts (shmmni segment exhaustion from leaks) and the still-modest msgmnb/msgmax for message queues. The second takeaway is the container leak boundary: relying on per-IPC-namespace teardown to reap leaked objects is robust only if the container actually has its own IPC namespace — docker run --ipc=host (or Kubernetes hostIPC: true) re-shares the host namespace and re-exposes both the host’s limits and the host-wide leak risk, so security- and reliability-conscious deployments avoid it.

See Also

Graph View

Backlinks