CPU Isolation isolcpus and nohz_full

CPU isolation is the practice of dedicating one or more processor cores to a single latency-critical thread and stripping away everything the kernel would otherwise schedule onto that core — the periodic scheduler-clock interrupt, unbound timers, Read-Copy-Update (RCU) callback processing, kernel worker threads, and the load balancer’s habit of migrating stray tasks onto an idle-looking CPU. It is configured almost entirely at boot through three kernel command-line parameters that cooperate: isolcpus= removes CPUs from the scheduler’s load-balancing domains, nohz_full= stops the periodic tick on a CPU that is running a single task (“full dynticks” / “adaptive-ticks”), and rcu_nocbs= offloads that CPU’s RCU callbacks to a kernel thread on another core (kernel-parameters.txt, v6.12; no_hz.rst, v6.12). The goal is to drive OS jitter — the unpredictable microsecond-to-millisecond stalls the kernel imposes on a running thread — as close to zero as the hardware allows, which is what high-performance computing (HPC), hard real-time control loops, and busy-polling packet-processing frameworks like the Data Plane Development Kit (DPDK) need. The single most important fact: these three knobs do different things, and getting low jitter usually requires all three plus a fourth absorber — the housekeeping CPUs that catch the displaced work.

This note pins behaviour to Linux 6.12 LTS, and dates everything later. As of 2026-09-04, kernel.org/releases.json reports mainline at 7.3-rc1, stable at 7.2.3, and five maintained longterm series — 6.18.49, 6.12.108, 6.6.156, 6.1.187, and 5.15.220. The v6.12 tag is dated 2024-11-17 and the v6.18 tag 2025-11-30 (git tagger timestamps via the GitHub tag API); v7.0 is dated 2026-04-12, v7.1 2026-06-14 and v7.2 2026-08-16. Reading v6.12 is therefore reading a maintained LTS, not the leading edge — and three things have materially changed since, each flagged inline where it belongs: the HK_FLAG_* consolidation in 6.14, the arrival of a dedicated Documentation/admin-guide/cpu-isolation.rst in 7.1, and the un-deprecation of isolcpus= in 7.2.

A boundary worth stating up front, because it is easy to blur: this note is about keeping work off a CPU. What the fair scheduler does with the work that remains — EEVDF’s eligibility test, virtual deadlines, base_slice, delayed dequeue, and the 6.12 per-run-queue fair_server that guarantees SCHED_OTHER bandwidth against real-time starvation — belongs to The EEVDF Scheduler and is not repeated here. Isolation is the negative space around that scheduler: isolcpus=domain removes a CPU from the load balancer’s reach so EEVDF on that run queue only ever has one entity to pick, and nohz_full then removes the tick that EEVDF would otherwise use to enforce preemption. The two notes meet exactly at sched_can_stop_tick(), which is scheduler code asked an isolation question.

Mental Model — Three Knobs, Three Noise Sources

The right way to think about CPU isolation is as a layered subtraction. A “normal” CPU under Linux carries three recurring obligations imposed by the kernel rather than by your workload, and each of the three boot parameters removes one of them:

flowchart TB
  subgraph NORMAL["A normal (housekeeping) CPU carries all three obligations"]
    LB["Load balancer may migrate<br/>other tasks onto this CPU"]
    TICK["Periodic scheduler tick<br/>(CONFIG_HZ, e.g. 250/1000 Hz)"]
    RCU["RCU callbacks run here<br/>in softirq context"]
  end
  subgraph ISO["An isolated CPU after all three knobs"]
    NLB["isolcpus= / isolcpus=domain<br/>→ removed from sched domains"]
    NTICK["nohz_full=<br/>→ tick stops when 1 task runs"]
    NRCU["rcu_nocbs= (implied by nohz_full)<br/>→ callbacks offloaded to rcuo* kthread"]
  end
  LB -.removes.-> NLB
  TICK -.removes.-> NTICK
  RCU -.removes.-> NRCU
  NLB --> RESID["Residual noise still present:<br/>1Hz remote tick, IPIs, NMIs,<br/>TLB shootdowns, page faults, unbound IRQs"]
  NTICK --> RESID
  NRCU --> RESID

The subtraction model of CPU isolation. What it shows: each boot parameter removes one category of kernel-imposed work from a CPU — isolcpus stops the load balancer from putting other tasks there, nohz_full stops the periodic tick, and rcu_nocbs (which nohz_full turns on automatically) stops RCU callbacks from running there. The insight to take: there is no single “isolate this CPU” switch. The three knobs are orthogonal — nohz_full does not remove the CPU from load balancing, and isolcpus does not stop the tick — and even with all three applied a layer of residual noise remains that no boot parameter can erase, which is why true low-jitter setups also pin interrupts and lock memory.

The Coverage Matrix — What Each Layer Does Not Cover

The mental-model diagram above is deliberately simplified. In practice there are five distinct isolation layers, not three, and the single most common configuration bug is assuming one of them subsumes another. Laying them out as a matrix makes the gaps explicit:

Layer	Interface	Removes	Explicitly does not cover	Reversible at runtime?
Scheduler-domain isolation	isolcpus=domain,<list> or cgroup v2 cpuset.cpus.partition=isolated	The CPU from SMP load balancing; as a side effect, from unbound workqueues and unbound kthreads	The tick; RCU callbacks; device IRQs; anything you pin there yourself	Boot param: no. Cpuset partition: yes
Tick isolation (full dynticks)	nohz_full=<list> or isolcpus=nohz,<list>	The periodic scheduler-clock interrupt, when the CPU has exactly one runnable task	Load balancing (HK_FLAG_DOMAIN is not in the nohz_full flag set); device IRQs; the residual 1 Hz remote tick	No
RCU callback offload	rcu_nocbs=<list> (implied by nohz_full)	RCU_SOFTIRQ callback invocation on that CPU	Grace-period detection work; the wakeups that hand callbacks to the rcuo kthreads (unless rcu_nocb_poll)	Partially — rcu_nocbs with no list allows runtime toggling via cpusets
Unmanaged IRQ affinity	irqaffinity=<housekeeping list>, /proc/irq/N/smp_affinity	Default delivery of ordinary device interrupts to isolated CPUs	Kernel-managed IRQs (multiqueue block, NVMe), whose affinity userspace cannot set; IPIs; NMIs	Yes, per-IRQ
Managed IRQ steering	isolcpus=managed_irq,<list>	Best-effort steering of kernel-managed queue interrupts to housekeeping CPUs	Queues whose affinity mask contains only isolated CPUs — then the flag has no effect at all	No

flowchart LR
  W["Kernel work aimed at CPU N"]
  W --> Q1{"Is it another<br/>runnable task?"}
  Q1 -->|yes| L1["isolcpus=domain<br/>or cpuset isolated partition"]
  Q1 -->|no| Q2{"Is it the periodic<br/>scheduler tick?"}
  Q2 -->|yes| L2["nohz_full=<br/>only if exactly 1 runnable task"]
  Q2 -->|no| Q3{"Is it an RCU<br/>callback?"}
  Q3 -->|yes| L3["rcu_nocbs=<br/>offload to rcuo* kthread"]
  Q3 -->|no| Q4{"Is it a device IRQ?"}
  Q4 -->|"yes, unmanaged"| L4["irqaffinity= /<br/>proc/irq/N/smp_affinity"]
  Q4 -->|"yes, kernel-managed"| L5["isolcpus=managed_irq<br/>best effort only"]
  Q4 -->|no| R["NOT COVERED BY ANY KNOB<br/>IPIs, TLB shootdowns, NMIs,<br/>SMIs, 1Hz remote tick,<br/>page faults, your own syscalls"]
  L1 --> OK["moved to a<br/>housekeeping CPU"]
  L2 --> OK
  L3 --> OK
  L4 --> OK
  L5 --> OK

Routing a unit of kernel work to the knob that removes it. What it shows: each isolation parameter answers exactly one question about the incoming work, and the questions are asked in series — nothing that falls through to the bottom is addressable by any boot parameter. The insight to take: the branch that matters is the last one. Four of the five layers move work to a housekeeping CPU; the fifth branch is a dead end, and everything in that box — inter-processor interrupts, TLB shootdowns, non-maskable interrupts, firmware System Management Interrupts, the residual 1 Hz remote tick, and the kernel work your own thread requests by making a syscall — is residual noise you must design around rather than configure away. “Fully isolated” is a configuration state, not a silence guarantee.

isolcpus= — Remove CPUs From the Scheduler’s Domains

The oldest of the three knobs is isolcpus=. Its core effect is to take the named CPUs out of the scheduler’s load-balancing domains (see Scheduling Domains and CPU Topology). The periodic load balancer walks these domains looking to even out run-queue lengths across cores; a CPU that is not in any balancing domain will never be chosen as a migration target, so the kernel will not move arbitrary SCHED_OTHER tasks onto it (kernel-parameters.txt, v6.12). You still place your thread there explicitly with [[CPU Affinity and sched_setaffinity|sched_setaffinity]] or a cpuset; isolation only stops the kernel from adding uninvited company.

The v6.12 documentation gives the format as isolcpus=[flag-list,]<cpu-list> and supports three flags, parsed in kernel/sched/isolation.c by housekeeping_isolcpus_setup():

• domain — isolate from SMP load balancing and scheduling. This is the default if no flag is given: the code does if (!flags) flags |= HK_FLAG_DOMAIN; (isolation.c, v6.12). So bare isolcpus=2,3 means isolcpus=domain,2,3.
• nohz — also stop the tick on those CPUs (equivalent to listing them in nohz_full=). In v6.12 this set only HK_FLAG_TICK; in 6.18 the same flag sets the consolidated HK_FLAG_KERNEL_NOISE, and the documentation was updated to say outright that isolcpus=nohz “is equivalent to the nohz_full parameter” (kernel-parameters.txt, v6.18).
• managed_irq — keep kernel-managed device interrupts (those whose affinity the kernel sets automatically, like NVMe/multiqueue-block completion IRQs) off the isolated CPUs on a best-effort basis: if a queue’s interrupt mask contains both isolated and housekeeping CPUs, the kernel steers the interrupt to a housekeeping CPU so that I/O submitted elsewhere cannot disturb the isolated core (kernel-parameters.txt, v6.12).

Is isolcpus Deprecated? Yes at 6.12, No Any More

This deserves its own treatment, because “isolcpus is deprecated” is repeated everywhere and, as of 2026, is no longer true upstream.

At v6.12 the claim is correct and verifiable by reading the file. Documentation/admin-guide/kernel-parameters.txt at that tag carries the literal tag [Deprecated - use cpusets instead] on the second line of the isolcpus= entry, and the domain flag’s description reads:

Note that performing domain isolation this way is irreversible: it’s not possible to bring back a CPU to the domains once isolated through isolcpus. It’s strongly advised to use cpusets instead to disable scheduler load balancing through the “cpuset.sched_load_balance” file. It offers a much more flexible interface where CPUs can move in and out of an isolated set anytime. — kernel-parameters.txt, v6.12

The irreversibility is real and remains true in every release: once the boot parameter has pulled a CPU out of the scheduling domains, nothing brings it back short of a reboot. What changed is the editorial judgement about whether that makes the parameter deprecated.

Commit 75ff0feaa275, “Documentation/kernel-parameters: Remove “Deprecated” from isolcpus=” by Sebastian Andrzej Siewior (Linutronix), authored 2026-04-27 and committed by Jonathan Corbet on 2026-05-03, deletes the tag. Its reasoning is worth quoting because it settles the question rather than restating it:

The isolcpus= option has been marked as deprecated in 2017. Back then it was desired for the domain sub option to be configured dynamically at runtime instead using this boot command line which provides a static configuration. In the meantime this option was extended by other sub options which don’t have runtime counterpart or it does not make sense to provide one.

The deprecated part always referred to the default domain' sub option but it was not obvious. Also the reasoning behind the deprecation is sort of dubious: There is nothing wrong with a static configuration if there is no desired to reconfigure. This is useful on systems which have one purpose and the CPU partition configuration is not changed for the entire lifetime. — commit [75ff0feaa275`](https://api.github.com/repos/torvalds/linux/commits/75ff0feaa275), Acked-by Waiman Long (the cpuset maintainer) and Steven Rostedt

Measured by fetching kernel-parameters.txt at each tag and counting the string Deprecated - use cpusets instead: present at v6.12, v6.13, v6.14, v6.15, v6.16, v6.17, v6.18, v7.0 and v7.1; absent at v7.2. The same commit also rewrites the domain text to drop the cpuset.sched_load_balance advice — which was itself stale, since that file is a cgroup v1 interface — and replace it with a pointer to the new Documentation/admin-guide/cpu-isolation.rst:

Note that performing domain isolation this way is irreversible: it’s not possible to bring back a CPU to the domains once isolated through this boot time configuration. Use cpusets for a dynamic configuration which can be altered at runtime. — kernel-parameters.txt, v7.2

timeline
    title The deprecation status of isolcpus=
    2017 : "[Deprecated - use cpusets instead]" tag added
         : rationale — domain isolation should be dynamic
    v6.12 (2024-11-17) : tag present
                       : advice points at cgroup-v1 cpuset.sched_load_balance
    v7.1 (2026-06-14) : tag still present
                      : Documentation/admin-guide/cpu-isolation.rst added by Frederic Weisbecker
                      : new doc calls isolcpus=domain "a less flexible alternative", not deprecated
    v7.2 (2026-08-16) : tag REMOVED by commit 75ff0feaa275
                      : advice rewritten to point at cpu-isolation.rst
                      : nohz and managed_irq flags were never deprecated

Nine years of editorial position on one boot parameter. What it shows: the deprecation was added in 2017, survived unchanged through the whole 6.x line, and was deleted in 7.2 with the explicit finding that the reasoning behind it was “sort of dubious”. The insight to take: what to take from the note’s own claim in a v6.12-pinned reading is that the tag is there — so a 6.12-era system reads as deprecated — but the substance was always narrower than the tag implied. The deprecation only ever applied to the default domain sub-option; nohz and managed_irq have no runtime cpuset counterpart and were never covered by it. Anyone repeating “isolcpus is deprecated” as a blanket statement in 2026 is quoting a line the kernel has since deleted.

So the honest guidance, current as of 2026-09-04, is a split one. Use cpuset.cpus.partition=isolated when you want to reconfigure the isolated set at runtime, or when the partition is tenant-driven (a container orchestrator carving out cores per pod). Use isolcpus=domain without embarrassment when the machine has exactly one job for its whole life — the upstream position is now explicitly that “there is nothing wrong with a static configuration if there is no desired to reconfigure.” And use isolcpus=managed_irq regardless of which of those two you picked, because it has no cpuset equivalent. See Cpusets and CPU Partitioning for the partition mechanics.

nohz_full= — Stop the Tick on a Single-Task CPU

nohz_full=<cpu-list> marks the listed CPUs as adaptive-ticks (also called full-dynticks) CPUs. The motivation is OS jitter: on a normal CPU a timer interrupt fires CONFIG_HZ times per second (commonly 250 or 1000) to drive the scheduler’s preemption and accounting. Each tick steals a few microseconds and, worse, evicts the running thread’s cache and TLB working set. The no_hz.rst documentation states the rule precisely: “If a CPU has only one runnable task, there is little point in sending it a scheduling-clock interrupt because there is no other task to switch to” (no_hz.rst, v6.12). So when a nohz_full CPU drops to exactly one runnable task, the kernel stops the periodic tick on it; the thread then runs uninterrupted (by the tick) until it blocks, exits, or a second task becomes runnable there. The exact “can the tick stop” decision lives in the housekeeping note (sched_can_stop_tick() / can_stop_full_tick()), so it is not repeated here — the short version is “the CPU must have at most one runnable task, with extra rules for the real-time and deadline classes.”

nohz_full requires CONFIG_NO_HZ_FULL=y to be compiled in; if the option is off, the setup code prints "Housekeeping: nohz unsupported. Build with CONFIG_NO_HZ_FULL" and ignores the parameter (isolation.c, v6.12). CONFIG_NO_HZ_FULL itself selects CONFIG_NO_HZ_COMMON, so adaptive ticks always implies dyntick-idle (tickless when idle) as well (no_hz.rst, v6.12).

Two cooperating side effects make nohz_full more than just a tick switch. First, the boot CPU “will be forced outside the range to maintain the timekeeping” — you cannot make CPU 0 adaptive-ticks, because at least one CPU must keep ticking to keep gettimeofday() accurate (kernel-parameters.txt, v6.12). Second, nohz_full automatically offloads RCU callbacks for the listed CPUs: “Any CPUs in this list will have their RCU callbacks offloaded, just as if they had also been called out in the rcu_nocbs= boot parameter” (kernel-parameters.txt, v6.12). That is necessary because a CPU with pending RCU callbacks cannot stay tickless — RCU needs the tick to make grace-period progress — so RCU offload is a precondition for a CPU to actually go quiet.

Inside the kernel, housekeeping_nohz_full_setup() translates nohz_full= into a set of housekeeping flags. In v6.12 that set is HK_FLAG_TICK | HK_FLAG_WQ | HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC | HK_FLAG_KTHREAD (isolation.c, v6.12). Note what is absent from that list: HK_FLAG_DOMAIN. This is the source of the single most common misconception about these knobs (covered below): nohz_full does not perform domain isolation. The HK_TYPE_* flag machinery itself, and the 6.14-era consolidation that collapsed those six flags into one, are covered in the housekeeping note.

The Condition People Miss: Exactly One Runnable Task

Everything about nohz_full hinges on a predicate most write-ups compress into “the CPU is busy.” It is not that. The tick stops only when the run queue is down to a single runnable entity, and the rule is class-by-class. sched_can_stop_tick() in kernel/sched/core.c, v6.12 is short enough to reason about completely:

bool sched_can_stop_tick(struct rq *rq)
{
	/* Deadline tasks, even if single, need the tick */
	if (rq->dl.dl_nr_running)
		return false;
	/* If there are more than one RR tasks, we need the tick to
	 * affect the actual RR behaviour. */
	if (rq->rt.rr_nr_running)
		return rq->rt.rr_nr_running == 1;
	/* If there's no RR tasks, but FIFO tasks, we can skip the tick,
	 * no forced preemption between FIFO tasks. */
	if (rq->rt.rt_nr_running - rq->rt.rr_nr_running)
		return true;
	if (scx_enabled() && !scx_can_stop_tick(rq))
		return false;
	if (rq->cfs.nr_running > 1)
		return false;
	if (__need_bw_check(rq, rq->curr))
		if (cfs_task_bw_constrained(rq->curr))
			return false;
	return true;
}

Reading it line by line, in the order the kernel checks:

1. Any SCHED_DEADLINE task at all forces the tick on. Not “more than one” — any. The deadline class needs the tick to enforce runtime budgets and replenishment periods, so a single SCHED_DEADLINE thread on a nohz_full CPU still gets ticked. This surprises people who reach for SCHED_DEADLINE because they want determinism.
2. SCHED_RR is counted separately from SCHED_FIFO. Round-robin needs the tick to expire timeslices, so two or more SCHED_RR threads keep it; exactly one may drop it.
3. SCHED_FIFO never needs the tick, however many threads there are — FIFO has no forced preemption between equal-priority threads, so there is no timeslice to expire. This is why the classic hard-real-time recipe is SCHED_FIFO, not SCHED_RR and not SCHED_DEADLINE.
4. For SCHED_OTHER (i.e. EEVDF) the test is rq->cfs.nr_running > 1. One fair task: tick can stop. Two: it cannot, because EEVDF needs the tick to enforce involuntary preemption at the end of a slice. Note this counts runnable entities on this run queue, not threads in your process — a kernel thread that briefly wakes on the isolated CPU is enough to push nr_running to 2 and switch the tick back on for as long as it stays.
5. CFS bandwidth control (cpu.max in a cgroup) forces the tick on even for a single task, because the quota refill has to be enforced somewhere. Putting an isolated thread in a bandwidth-limited cgroup silently defeats nohz_full.
6. sched_ext (scx_enabled()) gets asked. As of 6.12 a loaded BPF scheduler can veto tick-stopping through scx_can_stop_tick().

Passing sched_can_stop_tick() is necessary but not sufficient. The tick machinery keeps a second, orthogonal gate: an atomic tick dependency mask, checked by can_stop_full_tick() in kernel/time/tick-sched.c, v6.12. There are four masks — global, per-CPU, per-task, and per-signal (i.e. per thread group) — and any bit set in any of them keeps the tick running:

static bool can_stop_full_tick(int cpu, struct tick_sched *ts)
{
	if (unlikely(!cpu_online(cpu)))               return false;
	if (check_tick_dependency(&tick_dep_mask))            return false;  /* global */
	if (check_tick_dependency(&ts->tick_dep_mask))        return false;  /* this CPU */
	if (check_tick_dependency(&current->tick_dep_mask))   return false;  /* this task */
	if (check_tick_dependency(&current->signal->tick_dep_mask)) return false; /* thread group */
	return true;
}

The six bits are enumerated in include/linux/tick.h, v6.12, and each one is a concrete way a “fully isolated” CPU gets its tick back:

Bit	TICK_DEP_BIT_…	Who sets it	What it means for you
0	POSIX_TIMER	A timer_create() CPU-time timer, or setitimer(ITIMER_PROF/ITIMER_VIRTUAL)	Arming any per-process CPU-clock timer pins the tick on for the whole thread group — this is a per-signal dependency, so one thread’s timer ticks all of them
1	PERF_EVENTS	The perf subsystem when events must be round-robined	Profiling your isolated thread can destroy the property you are profiling for
2	SCHED	The scheduler, via sched_can_stop_tick() returning false	The “more than one runnable task” case above
3	CLOCK_UNSTABLE	Timekeeping, when the clocksource is unreliable	An unstable TSC needing the clocksource watchdog defeats nohz_full entirely
4	RCU	RCU, when this CPU still owes grace-period progress	The reason RCU callbacks must be offloaded before the tick can stop
5	RCU_EXP	An expedited RCU grace period in flight	A transient, system-wide event — e.g. someone unloading a module — reaches into your isolated CPU

stateDiagram-v2
    direction LR
    [*] --> Ticking
    Ticking --> Evaluating : task switch, enqueue,<br/>dequeue, or IRQ exit
    Evaluating --> Ticking : sched_can_stop_tick() == false<br/>(2+ CFS tasks, any DL task,<br/>2+ RR tasks, cpu.max quota)
    Evaluating --> Ticking : can_stop_full_tick() == false<br/>(any TICK_DEP bit set in global,<br/>cpu, task or signal mask)
    Evaluating --> TickStopped : both gates pass
    TickStopped --> Ticking : second task becomes runnable<br/>(sets TICK_DEP_BIT_SCHED)
    TickStopped --> Ticking : POSIX CPU timer armed,<br/>perf event, expedited RCU
    TickStopped --> TickStopped : 1 Hz remote tick fires<br/>FROM a housekeeping CPU<br/>(sched_tick_remote)
    TickStopped --> [*] : CPU goes idle<br/>(hands off to dyntick-idle)
    note right of TickStopped
      The thread runs without a local
      timer interrupt. It is NOT free of
      interrupts: IPIs, TLB shootdowns,
      NMIs and unbound IRQs still land.
    end note

The tick state machine of a nohz_full CPU. What it shows: stopping the tick is not a boot-time state but a continuously re-evaluated decision, gated by two independent predicates — the scheduler’s runnable-task count and the six-bit tick dependency mask — and re-entered from Ticking on every scheduling event. The insight to take: the arrows back to Ticking are the operationally important ones. Nothing warns you when they fire. A monitoring agent that briefly wakes on your isolated core, a perf record you left running, an expedited RCU grace period from an unrelated rmmod, or a cpu.max you forgot was set on the container — each silently reverts the CPU to a fully ticking one while the boot parameters still claim it is isolated. This is precisely why the verification section below exists: nohz_full= in /proc/cmdline proves configuration, not behaviour.

rcu_nocbs= — Offload the Callbacks

rcu_nocbs=[<cpu-list>] puts the listed CPUs into no-callbacks (no-CB) mode, requiring CONFIG_RCU_NOCB_CPU=y. Normally, when a CPU completes an RCU grace period it runs the deferred callbacks (the functions queued via call_rcu(), typically to free memory once no reader can still hold a reference) in RCU_SOFTIRQ context — directly on that CPU. That softirq is OS jitter. In no-CB mode the callbacks are instead handed to dedicated kernel threads named rcuop/N, rcuos/N, and rcuog/N (the suffixes denote RCU-preempt, RCU-sched, and the grace-period-mediating kthread respectively), which can be pinned to housekeeping CPUs (kernel-parameters.txt, v6.12; no_hz.rst, v6.12). The documentation states the payoff directly: “This reduces OS jitter on the offloaded CPUs, which can be useful for HPC and real-time workloads” (kernel-parameters.txt, v6.12).

There are two ways rcu_nocbs matters in practice. If you pass nohz_full=, you usually do not need to also pass rcu_nocbs= — nohz_full offloads RCU for the same CPUs automatically, and “Note that this argument takes precedence over the CONFIG_RCU_NOCB_CPU_DEFAULT_ALL option” (kernel-parameters.txt, v6.12). You would specify rcu_nocbs= independently only to offload callbacks from CPUs that are not full-dynticks (for example, to reduce jitter on a CPU that still takes the periodic tick but should never run callback work). The companion rcu_nocb_poll knob makes the rcuo kthreads poll for callbacks instead of waiting to be woken — improving the offloaded CPU’s latency by sparing it the wakeup, at the cost of the kthreads spinning and burning energy (kernel-parameters.txt, v6.12). The kernel does not pin the rcuo kthreads for you — the docs note “it is up to userspace to pin the ‘rcuo’ kthreads to specific CPUs if desired” (no_hz.rst, v6.12).

Why Isolation Without RCU Offload Still Gets RCU Work

The dependency runs in one direction and it is worth drawing, because “I set nohz_full, why is the tick still on?” is very often an RCU answer. A CPU that has queued RCU callbacks owes the grace-period machinery periodic progress reports; that progress is driven from the tick. So TICK_DEP_BIT_RCU gets set, can_stop_full_tick() returns false, and the tick never stops — no matter how correctly you configured the scheduler side. RCU offload is not a nice-to-have alongside tick isolation; it is a precondition for it, which is exactly why nohz_full= turns it on for you.

flowchart TB
  subgraph DEF["Default: callbacks run where they were queued"]
    direction TB
    CR["Kernel code on CPU 4 calls<br/>call_rcu(&amp;obj-&gt;rcu, free_obj)"]
    CQ["Callback queued on CPU 4's<br/>per-CPU rcu_data list"]
    GP["Grace period ends"]
    SI["RCU_SOFTIRQ runs ON CPU 4<br/>invokes the callbacks"]
    DEP["CPU 4 owes GP progress<br/>=&gt; TICK_DEP_BIT_RCU set<br/>=&gt; tick CANNOT stop"]
    CR --> CQ --> GP --> SI
    CQ --> DEP
  end
  subgraph OFF["rcu_nocbs=4 (implied by nohz_full=4)"]
    direction TB
    CR2["Kernel code on CPU 4 calls<br/>call_rcu()"]
    CQ2["Callback queued on CPU 4's<br/>no-CB list, then handed off"]
    KT["rcuog/N mediates the grace period<br/>rcuop/N and rcuos/N invoke callbacks<br/>— all schedulable kthreads"]
    HK["Pinned by USERSPACE to<br/>housekeeping CPUs 0-1"]
    OK2["CPU 4 owes RCU nothing<br/>=&gt; TICK_DEP_BIT_RCU clear<br/>=&gt; tick can stop"]
    CR2 --> CQ2 --> KT --> HK
    CQ2 --> OK2
  end
  DEF -.->|"rcu_nocbs= flips this"| OFF

RCU callback processing with and without offload. What it shows: call_rcu() does not run the callback; it queues it. Without offload the deferred work is invoked in RCU_SOFTIRQ on the very CPU you are trying to keep quiet, and that CPU’s outstanding grace-period obligation sets the RCU tick-dependency bit. With offload the work becomes an ordinary schedulable kernel thread that can be moved anywhere. The insight to take: two separate wins, and people usually notice only the first. Offload removes the softirq jitter, yes — but the load-bearing effect is that it clears TICK_DEP_BIT_RCU, which is what makes tick-stopping possible at all. Note the last step is yours: the kernel creates the rcuo* kthreads but does not pin them, so an unpinned rcuop/4 can still be scheduled onto CPU 4 and put the noise right back.

The kthread naming in the v6.12 documentation is precise: callbacks go to rcuox/N kthreads, “where ‘x’ is ‘p’ for RCU-preempt, ‘s’ for RCU-sched, and ‘g’ for the kthreads that mediate grace periods; and ‘N’ is the CPU number” (kernel-parameters.txt, v6.12). On a machine with no offloading configured they do not exist at all, which is a fast way to check. On the 32-CPU AMD Ryzen AI MAX+ 395 box these notes were verified on — running Fedora’s 7.1.8 kernel with CONFIG_RCU_NOCB_CPU=y, CONFIG_RCU_NOCB_CPU_DEFAULT_ALL not set, and no rcu_nocbs= on the command line — ps -eo comm | grep -c '^rcuo' returns 0: the capability is compiled in but no CPU is in no-CB mode. Contrast that with ksoftirqd/0 … ksoftirqd/31, which are always present. If you have configured rcu_nocbs= and see no rcuo* threads, the parameter did not take effect.

Configuration — A Worked Isolation Setup

A typical low-jitter configuration on an 8-core machine that dedicates CPUs 2–7 to the workload and keeps CPUs 0–1 as housekeeping looks like this on the kernel command line (e.g. via GRUB GRUB_CMDLINE_LINUX):

isolcpus=domain,managed_irq,2-7 nohz_full=2-7 rcu_nocbs=2-7

Line by line:

• isolcpus=domain,managed_irq,2-7 — remove CPUs 2–7 from load-balancing domains (domain) so the scheduler never migrates uninvited tasks there, and keep kernel-managed device IRQs off them on a best-effort basis (managed_irq). This is the irreversible, boot-only part.
• nohz_full=2-7 — make CPUs 2–7 adaptive-ticks: the periodic scheduler interrupt stops on each whenever it is down to one runnable task. This forces CPU 0/1 to remain the timekeeping CPU(s) and implicitly offloads RCU for 2–7.
• rcu_nocbs=2-7 — explicit and redundant with nohz_full here, but commonly written out for clarity / belt-and-braces. The masks must match nohz_full/isolcpus=nohz: the setup code rejects a mismatch with "Housekeeping: nohz_full= must match isolcpus=" (isolation.c, v6.12).

After boot you still must do the userspace half: pin your latency thread to a single isolated CPU with taskset -c 4 ./worker or sched_setaffinity(), move the global unbound-workqueue mask off the isolated CPUs via /sys/devices/virtual/workqueue/cpumask, and pin the rcuo* kthreads to the housekeeping set. The workqueue step is explicitly called out in the docs: with isolcpus=nohz “a residual 1Hz tick is offloaded to workqueues, which you need to affine to housekeeping … the global workqueue runs on all CPUs, so to protect individual CPUs the ‘cpumask’ file has to be configured manually after bootup” (kernel-parameters.txt, v6.12).

You can confirm the active isolation by reading the read-only sysfs file /sys/devices/system/cpu/isolated (the effective isolated set) and /proc/cmdline.

The Upstream Reference Configuration

Since Linux 7.1 the kernel ships a dedicated document for this, Documentation/admin-guide/cpu-isolation.rst, added by commit “doc: Add CPU Isolation documentation” from Frederic Weisbecker (dated 2026-04-02; the file 404s at v7.0 and is present from v7.1). It is now the single best primary text on the subject, and it disagrees in an instructive way with the folklore recipe above: it does not use isolcpus=domain at all.

Its worked example isolates CPU 7 of an 8-CPU machine with the command line (cpu-isolation.rst, v7.2):

nohz_full=7 irqaffinity=0-6 isolcpus=managed_irq,7 nosmt

Element by element:

• nohz_full=7 — tick isolation for CPU 7, which also implies RCU callback offload for it, and forces CPU 0 to remain the timekeeper.
• irqaffinity=0-6 — the default IRQ affinity mask for ordinary (non-managed) device interrupts. New IRQs inherit this, so nothing lands on CPU 7 by default.
• isolcpus=managed_irq,7 — the only isolcpus flag used, because managed IRQ steering has no cpuset equivalent. Note the deliberate absence of domain.
• nosmt — disables simultaneous multithreading, because a sibling hardware thread sharing CPU 7’s core competes for execution resources and “preempts” it in a way no scheduler knob can see.

Domain isolation is then done at runtime with a cgroup v2 cpuset partition, which is the part of the recipe that has changed since the 6.12-era advice:

cd /sys/fs/cgroup
echo +cpuset > cgroup.subtree_control   # activate the cpuset controller
mkdir test && cd test
echo +cpuset > cgroup.subtree_control
echo 7 > cpuset.cpus                    # this partition owns CPU 7
echo "isolated" > cpuset.cpus.partition # root partition WITHOUT load balancing

Writing isolated (rather than root) is what does the work. Per cgroup-v2.rst, v6.12, cpuset.cpus.partition accepts three values — member, root, and isolated — and “when set to isolated, the CPUs in that partition will be in an isolated state without any load balancing from the scheduler and excluded from the unbound workqueues.” That last clause matters: it does for free the /sys/devices/virtual/workqueue/cpumask step you must do by hand with isolcpus. The set of CPUs currently in isolated partitions is readable from the root cgroup’s cpuset.cpus.isolated, and “all possible state transitions among member, root and isolated are allowed” — the reversibility that boot-time isolcpus=domain cannot offer. A partition root can also read back as "isolated invalid (<reason>)", a degraded state that behaves like member; a monitoring check that only greps for the string isolated will happily pass on a partition that has silently stopped isolating.

Boot Parameters and Their cgroup-v2 Equivalents

Boot parameter	What it does	cgroup v2 equivalent	Reversible	Notes
isolcpus=domain,<list>	Remove from sched domains	cpuset.cpus.partition = isolated on a cpuset owning <list>	Only the cpuset form	The cpuset form also excludes the CPUs from unbound workqueues
isolcpus=nohz,<list>	Stop the tick (alias for nohz_full=)	none	No	Tick isolation is boot-only; there is no cpuset knob
isolcpus=managed_irq,<list>	Best-effort steering of kernel-managed IRQs	none	No	Why the upstream recipe still passes isolcpus= even when using cpusets
nohz_full=<list>	Adaptive-ticks; implies RCU offload	none	No	Boot CPU is always forced out of the list
rcu_nocbs=<list>	Callback offload	Partial — rcu_nocbs with no list allows per-CPU toggling via cpusets at runtime	Partial	With a list, the mode is fixed at boot
irqaffinity=<list>	Default affinity for new IRQs	/proc/irq/default_smp_affinity (procfs, not cgroup)	Yes	Per-IRQ override via /proc/irq/N/smp_affinity
(none)	Exclude unbound workqueues	/sys/devices/virtual/workqueue/cpumask, or implied by an isolated partition	Yes	Must be done manually when using isolcpus

Reading the table: the right-hand column is the point. Only one of the six boot parameters has a full runtime equivalent. The narrative that cpusets have replaced the boot parameters is half true at best — they replaced isolcpus=domain and nothing else, which is exactly what the 7.2 un-deprecation commit says.

IRQ Isolation — the Layer Boot Parameters Half-Cover

Device interrupts are the noise source most likely to be forgotten, because unlike the tick they are invisible until the device is busy. There are three classes and they need three different treatments, laid out in Documentation/core-api/irq/irq-affinity.rst and the isolcpus entry:

1. Ordinary (unmanaged) device IRQs. Affinity is userspace’s to set. /proc/irq/<N>/smp_affinity takes a hex bitmask and /proc/irq/<N>/smp_affinity_list takes a CPU list; /proc/irq/default_smp_affinity supplies the mask that any newly allocated IRQ inherits, and its default is all-CPUs (0xffffffff). The irqaffinity=<cpu-list> boot parameter — documented tersely at v6.12 as “[SMP] Set the default irq affinity mask” — seeds that default at boot so you are not racing driver probe. It is not allowed to mask off every CPU, and “if an IRQ controller does not support IRQ affinity then the value will not change from the default of all CPUs” — a silent failure mode worth checking rather than assuming.
2. Kernel-managed IRQs. Multiqueue block, NVMe and modern NIC completion interrupts have their affinity assigned by the kernel to spread queues across CPUs, and the doc states flatly that this affinity “cannot be changed via the /proc/irq/* interfaces.” isolcpus=managed_irq,<list> asks the kernel to prefer housekeeping CPUs when a queue’s automatically assigned mask contains both kinds. It is explicitly best effort: “if a queue’s affinity mask contains only isolated CPUs then this parameter has no effect on the interrupt routing decision, though interrupts are only delivered when tasks running on those isolated CPUs submit IO.”
3. IPIs, NMIs and SMIs. Not device interrupts and not steerable at all. Covered in the residual-noise section below.

The practical consequence of (2) is a design rule, not a configuration: do not do I/O from an isolated CPU. The kernel’s own escape hatch is that a managed queue interrupt fires on an isolated CPU only when work was submitted from that CPU — so the isolation you actually get is the isolation your application’s I/O discipline gives you.

flowchart TB
  IRQ["A device raises an interrupt"] --> K{"Who owns the<br/>affinity mask?"}
  K -->|"userspace"| U["Unmanaged IRQ"]
  K -->|"the kernel"| M["Managed IRQ<br/>(NVMe, blk-mq, multiqueue NIC)"]
  U --> U1["irqaffinity=0-6 at boot<br/>seeds /proc/irq/default_smp_affinity"]
  U1 --> U2["/proc/irq/N/smp_affinity<br/>per-IRQ override at runtime"]
  U2 --> U3{"Controller supports<br/>affinity?"}
  U3 -->|no| UF["Mask SILENTLY ignored<br/>stays all-CPUs"]
  U3 -->|yes| OKU["Delivered to housekeeping"]
  M --> M1["isolcpus=managed_irq,7"]
  M1 --> M2{"Queue mask contains<br/>housekeeping CPUs too?"}
  M2 -->|yes| OKM["Steered to a housekeeping CPU"]
  M2 -->|"no — isolated only"| MF["No effect. Fires on CPU 7<br/>— but only if CPU 7 submitted the I/O"]
  OKU --> V["Verify in /proc/interrupts:<br/>the isolated column must stay flat"]
  OKM --> V
  UF --> V
  MF --> V

The two ownership regimes for interrupt affinity. What it shows: whether you can move an interrupt away from a CPU depends on who assigned its mask, and there are two distinct silent-failure branches — a controller that ignores smp_affinity, and a managed queue whose mask contains only isolated CPUs. The insight to take: every path converges on the same verification step. There is no configuration you can write that guarantees an interrupt-free CPU; the only thing that establishes it is reading the per-CPU columns of /proc/interrupts before and after a load test and confirming the isolated column did not move.

Why HPC, Real-Time, and DPDK Use This

Three workload classes drive almost all CPU-isolation deployments, and the kernel’s own documentation names the first two:

• HPC with short, synchronized iterations. In a tightly coupled parallel job, all ranks proceed in lockstep at a barrier. The no_hz.rst text spells out the multiplier: “If any CPU is delayed during a given iteration, all the other CPUs will be forced to wait idle while the delayed CPU finishes. Thus, the delay is multiplied by one less than the number of CPUs” (no_hz.rst, v6.12). A single 5 µs tick on one rank of a 1000-rank job can cost ~5 ms of aggregate wall time. Removing the tick removes that tail.
• Real-time control loops. Adaptive ticks let an application “improve their worst-case response times by the maximum duration of a scheduling-clock interrupt” (no_hz.rst, v6.12). For deterministic latency you usually combine isolation with PREEMPT_RT and run the thread as SCHED_FIFO.
• Busy-polling packet processing (DPDK and similar). A DPDK poll-mode driver runs an infinite loop on a dedicated core polling NIC receive queues, never sleeping and never making a system call on the fast path. It is the canonical “one runnable task forever” workload — exactly what nohz_full is built for. Any tick, callback, or migrated task on that core directly shows up as dropped packets or latency spikes.

sequenceDiagram
    autonumber
    participant R0 as Rank 0 (CPU 0)
    participant R1 as Rank 1 (CPU 1)
    participant R2 as Rank 2 (CPU 2)
    participant B as MPI barrier
    Note over R0,R2: Iteration N — all ranks compute the same amount of work
    R0->>B: arrive at t = 100 us
    R2->>B: arrive at t = 100 us
    Note over R1: local timer tick fires<br/>~5 us stolen + cache and TLB evicted
    R1->>B: arrive at t = 105 us
    B-->>R0: release at t = 105 us
    B-->>R1: release at t = 105 us
    B-->>R2: release at t = 105 us
    Note over R0,R2: R0 and R2 each sat idle for 5 us.<br/>One tick on ONE rank cost 5 us x (N-1) of aggregate CPU.<br/>At N = 1000 ranks that is ~5 ms of wasted machine time,<br/>every time any single rank is ticked.

Why the tick is disproportionately expensive in tightly coupled parallel jobs. What it shows: a synchronising barrier converts a delay on one rank into idle time on every other rank, so the cost of a single interrupt is multiplied rather than amortised. no_hz.rst states the rule directly: “If any CPU is delayed during a given iteration, all the other CPUs will be forced to wait idle while the delayed CPU finishes. Thus, the delay is multiplied by one less than the number of CPUs.” The insight to take: this is why HPC cares about jitter rather than throughput overhead. A 1000 Hz tick costs each core well under 1% of its cycles — negligible if you only look at one core. Across a 1000-rank barrier-synchronised job, the same tick means some rank is being delayed essentially all the time, and the job runs at the pace of whichever rank was unlucky. The metric that matters is the tail, and isolation is a tail-latency intervention.

Residual Noise — What Isolation Cannot Remove

This is the honest centre of the topic. nohz_full does not mean no interrupts; it means no periodic scheduler-clock interrupt from this CPU’s own local timer, under conditions. Everything else a CPU can be interrupted by is still there. Drawing the survivors is more useful than any amount of prose about how quiet an isolated core is, so here they are, grouped by whether anything at all can be done about them:

flowchart TB
  CPU["CPU 7: isolcpus=managed_irq,7 nohz_full=7<br/>cpuset isolated partition, one SCHED_FIFO thread"]

  subgraph GONE["Removed by configuration"]
    G1["Periodic local timer tick<br/>(CONFIG_HZ, 250-1000 Hz)"]
    G2["RCU_SOFTIRQ callback invocation"]
    G3["Migrated SCHED_OTHER tasks"]
    G4["Unbound workqueue items"]
    G5["Unbound kthreads and timers"]
    G6["Default-affinity device IRQs"]
  end

  subgraph MITIG["Reducible, never eliminated"]
    M1["1 Hz REMOTE tick<br/>sched_tick_remote() from a<br/>housekeeping CPU, requeued at HZ<br/>— still present in v7.2"]
    M2["Managed IRQs for I/O this CPU<br/>submitted itself"]
    M3["Page faults<br/>— mlockall() and pre-fault"]
    M4["Kernel entry/exit cost on every<br/>syscall (RCU EQS bookkeeping,<br/>on-boundary CPU-time accounting)"]
    M5["SMT sibling contention<br/>— nosmt"]
    M6["Cpufreq transitions and<br/>deep C-state wakeup latency"]
  end

  subgraph HARD["Not addressable by any kernel knob"]
    H1["Rescheduling IPIs (RES)"]
    H2["Function-call IPIs (CAL)<br/>smp_call_function on all CPUs"]
    H3["TLB shootdown IPIs (TLB)<br/>from unmap/rmmod ANYWHERE"]
    H4["irq_work IPIs (IWI)"]
    H5["NMIs and machine-check polls"]
    H6["Firmware SMIs<br/>— BIOS-level, kernel-invisible"]
    H7["Expedited RCU grace periods<br/>TICK_DEP_BIT_RCU_EXP"]
    H8["POSIX CPU timers and perf events<br/>YOU armed<br/>— force the tick back on"]
  end

  CPU --> GONE
  CPU --> MITIG
  CPU --> HARD
  MITIG --> J["Measured jitter on the isolated CPU"]
  HARD --> J

What is still hitting a “fully isolated” CPU. What it shows: three tiers. Configuration genuinely removes the top box — that is real and it is most of the win. The middle box is reduced by discipline rather than by parameters (lock your memory, do not do I/O, do not make syscalls, disable SMT, pin C-states). The bottom box is the part nobody can configure away: any CPU on the machine that unmaps memory or unloads a module sends your isolated core a TLB-shootdown IPI, and firmware SMIs are not even visible to the kernel. The insight to take: the residual sources are dominated by things triggered elsewhere on the machine, not on the isolated CPU. This inverts the usual mental model. Tuning the isolated CPU is the easy half; the hard half is constraining what the rest of the system is allowed to do — which is why serious deployments dedicate whole machines, not just cores.

Two entries in that diagram deserve the record straight, because both are commonly stated wrongly.

The 1 Hz remote tick is still there. It is frequently described as a transitional wart that “was removed in the 6.x series.” Measured on 2026-09-04 by fetching kernel/sched/core.c at three tags: sched_tick_remote() is present at v6.12, v6.18 and v7.2, and in all three the function ends by re-queueing itself:

	/*
	 * Run the remote tick once per second (1Hz). This arbitrary
	 * frequency is large enough to avoid overload but short enough
	 * to keep scheduler internal stats reasonably up to date.
	 */
	os = atomic_fetch_add_unless(&twork->state, -1, TICK_SCHED_REMOTE_RUNNING);
	if (os == TICK_SCHED_REMOTE_RUNNING)
		queue_delayed_work(system_unbound_wq, dwork, HZ);

The important subtlety is where it runs. The work item goes on system_unbound_wq, so on a correctly configured system it executes on a housekeeping CPU and reaches into the isolated run queue under rq_lock_irq to call task_tick() and calc_load_nohz_remote(). The isolated CPU is not interrupted by a timer — it is interrupted by lock contention and cache-line traffic on its own run queue, once a second. Documentation/admin-guide/cpu-isolation.rst lists it as a permanent tradeoff, not a bug: “Housekeeping CPUs must run a 1Hz residual remote scheduler tick on behalf of the isolated CPUs.” If the unbound workqueue mask has not been narrowed — which, with isolcpus but without an isolated cpuset partition, it has not, since the global mask defaults to all CPUs — the remote tick can be dispatched onto the isolated CPU itself, at which point it is a genuine local interruption. That is the entire reason the isolcpus=nohz documentation shouts about /sys/devices/virtual/workqueue/cpumask.

Syscalls are expensive on a nohz_full CPU, not merely unhelpful. The upstream doc is blunt about the cost model: kernel entry and exit “are more costly due to fully ordered RmW operations that maintain userspace as RCU extended quiescent state. Also the CPU time is accounted on kernel boundaries instead of periodically from the tick” (cpu-isolation.rst, v7.2). Because there is no tick to sample CPU time, every transition has to do the accounting itself. The rule “no call to the kernel from isolated CPUs” is listed there as a constraint, alongside “the isolated CPUs must run a single task only” and “no use of POSIX CPU timers” — not as advice.

The no_hz.rst “Known Issues” section is the older authoritative enumeration, and its items map onto the diagram as follows (no_hz.rst, v6.12):

• The residual ~1 Hz remote tick. Some scheduler bookkeeping — load average, sched averages, CFS/EEVDF vruntime, avenrun, load balancing — “currently accommodated by scheduling-clock tick every second or so.” Mechanically this is a remote tick driven from a housekeeping CPU; the implementation (sched_tick_remote, the unbound-workqueue work re-queued at HZ) is detailed in Housekeeping CPUs and Tickless Isolation.
• Inter-processor interrupts (IPIs) and TLB shootdowns. A global TLB shootdown (triggered by, e.g., unmapping memory or unloading a kernel module elsewhere) sends an IPI to all CPUs including isolated ones. The only cure is “to avoid the unmapping operations … that result in these shootdowns” (no_hz.rst, v6.12).
• Page faults. Faulting in a page is kernel work on the isolated CPU; pre-fault and mlockall() the working set to avoid it.
• POSIX CPU timers force the tick back on. “POSIX CPU timers prevent CPUs from entering adaptive-tick mode” — if your thread arms a per-process CPU-time timer, the tick returns (no_hz.rst, v6.12).
• Perf events. Too many pending hardware perf events may pull the CPU out of adaptive-ticks for round-robining (no_hz.rst, v6.12).
• Unbound and non-managed IRQs, NMIs, machine-check exceptions — outside the kernel’s affinity control or unavoidable in hardware.

Uncertain

Verify: the quantitative residual-jitter figures a tuned isolated CPU achieves on modern hardware (often cited as sub-microsecond to single-digit-microsecond worst case). Reason: the kernel documentation describes the mechanism of residual noise thoroughly but publishes no numbers, and the figure is hardware-, firmware- and workload-specific — an SMI-happy BIOS alone can dominate it. No primary measurement was consulted for this note, and the reference machine available here (a single-socket 32-CPU laptop with no isolation configured) cannot produce one. To resolve: run rtla osnoise on an actually isolated core, or Frederic Weisbecker’s dynticks-testing suite (git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git), both named by cpu-isolation.rst, v7.2; or cite a vendor RT-tuning measurement report and state its hardware. uncertain

Verifying Isolation Instead of Assuming It

Everything above is configuration. None of it is evidence. The gap between “I passed nohz_full=” and “the tick is actually stopped on that CPU right now” is where most CPU-isolation disappointment lives, and closing it is the practically useful half of this topic. There are four levels of check, cheapest first.

flowchart TB
  L0["Level 0 — was it parsed?<br/>/proc/cmdline<br/>/sys/devices/system/cpu/isolated<br/>/sys/devices/system/cpu/nohz_full<br/>dmesg | grep -i housekeeping"]
  L1["Level 1 — is anything landing?<br/>Snapshot /proc/interrupts and<br/>/proc/softirqs, run the workload,<br/>snapshot again, diff the isolated column"]
  L2["Level 2 — why did the tick come back?<br/>trace the tick_stop tracepoint;<br/>its arg carries the TICK_DEP bit"]
  L3["Level 3 — what preempted me?<br/>sched:sched_switch + irq_vectors<br/>on the isolated CPU only,<br/>or rtla osnoise"]

  L0 --> Q0{"Masks non-empty<br/>and as intended?"}
  Q0 -->|no| F0["Parameter typo, CONFIG missing,<br/>or nohz_full=/isolcpus= mask mismatch<br/>— check dmesg for the pr_warn"]
  Q0 -->|yes| L1
  L1 --> Q1{"Isolated column<br/>delta == 0?"}
  Q1 -->|no| F1["Identify the row: LOC = tick back on,<br/>TLB/CAL = someone else's shootdown,<br/>a device row = IRQ affinity gap"]
  Q1 -->|yes| L2
  L2 --> Q2{"tick_stop events<br/>with a nonzero dep bit?"}
  Q2 -->|yes| F2["Map the bit: 0 POSIX_TIMER, 1 PERF_EVENTS,<br/>2 SCHED, 3 CLOCK_UNSTABLE,<br/>4 RCU, 5 RCU_EXP"]
  Q2 -->|no| L3
  L3 --> DONE["Residual noise characterised.<br/>Anything left is the bottom tier:<br/>IPIs, NMIs, SMIs, firmware"]

The isolation verification ladder. What it shows: four checks that get progressively more expensive and more specific, each with a defined failure branch that names the next thing to look at. The insight to take: never skip Level 0. Two of the most common failures — a CONFIG_NO_HZ_FULL=n kernel silently ignoring nohz_full=, and mismatched nohz_full=/isolcpus= masks — produce a pr_warn at boot and an empty sysfs mask, and cost nothing to detect. People routinely go straight to tracing and spend a day on a problem that cat /sys/devices/system/cpu/nohz_full would have answered.

Level 0 — did the kernel accept the parameters?

Two read-only sysfs files report the effective masks rather than what you typed:

$ cat /proc/cmdline
$ cat /sys/devices/system/cpu/isolated     # effective isolcpus= domain set
$ cat /sys/devices/system/cpu/nohz_full    # effective adaptive-ticks set
$ cat /sys/devices/virtual/workqueue/cpumask   # unbound workqueue mask
$ cat /sys/fs/cgroup/cpuset.cpus.isolated  # CPUs in isolated cpuset partitions

On the 32-CPU AMD Ryzen AI MAX+ 395 laptop these notes were verified against (Fedora kernel 7.1.8-200.fc44.x86_64, CONFIG_HZ_1000=y, CONFIG_NO_HZ_FULL=y, CONFIG_RCU_NOCB_CPU=y, CONFIG_PREEMPT_RT not set), the answer is the baseline case — nothing isolated:

$ cat /proc/cmdline
BOOT_IMAGE=(hd0,gpt2)/vmlinuz-7.1.8-200.fc44.x86_64 root=UUID=... ro rootflags=subvol=root
rd.luks.uuid=luks-... rhgb quiet
 
$ cat /sys/devices/system/cpu/isolated     # (empty)
$ cat /sys/devices/system/cpu/nohz_full    # (empty)
$ cat /sys/devices/virtual/workqueue/cpumask
ffffffff

Note that CONFIG_NO_HZ_FULL=y is set and yet nohz_full is empty — the capability is compiled in and unused. This is the normal state of a general-purpose distribution kernel and the reason “my distro supports it” is not the same claim as “my CPU is isolated.” A ffffffff workqueue mask means unbound work may be dispatched to any of the 32 CPUs, including any you later isolate with isolcpus.

The corresponding boot-time warnings, all from kernel/sched/isolation.c, v6.12, are worth grepping dmesg for by name:

dmesg string	Meaning
Housekeeping: nohz unsupported. Build with CONFIG_NO_HZ_FULL	nohz_full=/isolcpus=nohz silently ignored
Housekeeping: nohz_full= or isolcpus= incorrect CPU range	cpulist_parse() failed — malformed list
Housekeeping: must include one present CPU, using boot CPU:N	You tried to isolate every CPU; the kernel took one back
Housekeeping: nohz_full= must match isolcpus=	The two masks disagree; the second parameter is dropped
isolcpus: Invalid flag <x>	Non-alphabetic character in a flag — the whole parameter is discarded
isolcpus: Skipped unknown flag <x>	Flag ignored but parsing continues (e.g. a flag from a newer kernel)

The mask-mismatch case is the nastiest, because it is a partial failure: the first parameter takes effect and the second does not, so you get domain isolation without tick isolation (or the reverse) and no other symptom.

Level 1 — read the per-CPU interrupt counters

/proc/interrupts has one column per CPU. The technique is a diff, not an absolute reading: snapshot, run the workload for a fixed interval, snapshot again, and subtract the isolated CPU’s column. On a properly isolated core running a busy-poll thread, every row’s delta should be zero or near it — most importantly the LOC row, which counts local timer interrupts.

Summed across all 32 CPUs on the unisolated reference machine (uptime measured in weeks), the architectural rows read:

Row	Total across 32 CPUs	What it is	Isolation-relevant?
CAL	11,987,392,655	Function-call IPIs (smp_call_function*)	Not removable — cross-CPU work requests
LOC	9,706,354,376	Local timer interrupts	This is the tick. nohz_full targets exactly this row
RES	1,007,990,210	Rescheduling IPIs	Reduced by domain isolation, never zero
TLB	825,726,348	TLB shootdowns	Not removable — triggered by unmaps anywhere
IWI	92,740,140	irq_work IPIs (deferred work from NMI/IRQ context)	Not removable
NMI / PMI	232,171 / 232,171	Non-maskable / performance-monitoring interrupts	Identical counts: these are perf’s NMIs
MCP	74,368	Machine-check polls	Not removable
SPU, THR, DFR, ERR, MIS	0	Spurious, thermal-threshold, deferred-error APIC, error, mis-routed	Healthy system

The LOC figure is the one to internalise: with CONFIG_HZ=1000 and 32 CPUs, this box has taken nearly ten billion timer interrupts. nohz_full is a bid to take a slice of that number to zero on the cores that matter. Equally instructive is that CAL — pure cross-CPU IPI traffic, entirely outside the reach of any isolation parameter — is larger than LOC. Removing the tick does not make a CPU the quietest thing on the machine; it makes it quiet with respect to one specific source.

/proc/softirqs is the complementary view, and the row that matters for RCU offload is RCU. Same machine, CPU 0 through CPU 3:

                    CPU0           CPU1           CPU2           CPU3
TIMER:          28612497       15590656       21551453       16353432
NET_RX:         11528979        6084244       11683106        8654591
BLOCK:            104904          37864          69679          48916
TASKLET:       124064233       80386088      123284954      129453879
SCHED:         350611274      208598601      137698045      107096541
HRTIMER:           96445          46269         106107         110585
RCU:           154148487      104054096      139931210      117284345

On a CPU listed in rcu_nocbs= the RCU column should be flat: the callbacks are being invoked in rcuop/N instead of in softirq context. A growing RCU count on a supposedly offloaded CPU means the parameter did not take. Likewise a growing BLOCK or NET_RX count on an isolated CPU is the managed-IRQ gap from the previous section made visible.

Level 2 — trace why the tick came back

When Level 1 shows a nonzero LOC delta, the question is which of the six dependency bits is responsible. The kernel emits a tick_stop tracepoint on every decision, and check_tick_dependency() calls trace_tick_stop(0, TICK_DEP_MASK_<X>) with the specific bit that blocked it, while a successful stop traces trace_tick_stop(1, TICK_DEP_MASK_NONE) (tick-sched.c, v6.12). So the tracepoint’s arguments are a direct readout of the table above:

cd /sys/kernel/tracing
echo 1 > events/timer/tick_stop/enable
echo 1 > tracing_on
# ... run the workload ...
echo 0 > tracing_on
cat per_cpu/cpu7/trace

A line with success=0 and dependency=SCHED means a second task was runnable. dependency=POSIX_TIMER means somebody in your thread group armed a CPU-time timer. dependency=RCU means offload is not working. This maps a symptom to a cause in one step and is the single highest-value debugging technique in the topic.

Level 3 — characterise what is left

Documentation/admin-guide/cpu-isolation.rst (v7.1+) supplies a complete reproducible harness for this, and it is worth using verbatim rather than inventing one. The shape is: enable sched:sched_switch (to catch any task preempting yours) and the irq_vectors event group (to catch every IPI and interrupt vector), run a pure userspace spin loop pinned to the isolated CPU for ten seconds, then read per_cpu/cpu7/trace. The upstream document shows what a clean result looks like — a single sched_switch from the idle task into the workload, and nothing else for ten seconds except a pair of reschedule_entry/reschedule_exit events:

<idle>-0 [007] d..2. 1980.976624: sched_switch: prev_comm=swapper/7 ... ==> next_comm=user_loop next_pid=1553
user_loop-1553 [007] d.h.. 1990.946593: reschedule_entry: vector=253
user_loop-1553 [007] d.h.. 1990.946593: reschedule_exit: vector=253

Two higher-level tools sit above the raw tracing, both named by the upstream doc:

• rtla osnoise — part of the in-tree Real-Time Linux Analysis suite (Documentation/tools/rtla/), a kernel tracer that runs a measurement thread and reports a summary of the OS noise it observed, broken down by source.
• dynticks-testing — Frederic Weisbecker’s userspace equivalent, at git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git, and the suite no_hz.rst has always referred to.

The doc’s own checklist for what to fix when noise persists is short and mostly not about the isolation parameters at all: use mlock() so page faults cannot occur; avoid SMT so a sibling hyperthread cannot steal execution resources; tune cpufreq carefully because frequency transitions are themselves jitter; limit deep C-states via processor.max_cstate / intel_idle.max_cstate because exiting one has real wake-up latency; and check the BIOS for System Management Interrupts, which “your vendor will have a BIOS tuning guidance for” if you are lucky. Every one of those is outside the scheduler.

Common Misunderstandings

• “nohz_full isolates the CPU from the scheduler.” No. nohz_full stops the tick; it does not set HK_FLAG_DOMAIN, so the load balancer can still migrate other tasks onto a nohz_full-only CPU. The moment a second runnable task lands there, the tick comes back and your thread starts getting preempted. You must also pass isolcpus=domain (or use a cpuset isolated partition) to keep the CPU clear. This is provable directly from housekeeping_nohz_full_setup(), whose flag set omits HK_FLAG_DOMAIN (isolation.c, v6.12).
• “isolcpus stops the tick.” No. Bare isolcpus= defaults to domain only — load-balancing isolation. You need the nohz flag (isolcpus=nohz,...) or a separate nohz_full= to stop the tick.
• “You can nohz_full all CPUs.” No. At least one CPU must keep timekeeping; the boot CPU is forced out of the nohz_full range, and CONFIG_NO_HZ_FULL does nothing useful on a single-CPU system (no_hz.rst, v6.12).
• “Isolation gives a fully silent CPU.” No — see the residual-noise list above. Isolation reduces jitter dramatically but does not eliminate it.
• “isolcpus is deprecated, always use cpusets.” True at v6.12, false as of v7.2 — the [Deprecated - use cpusets instead] tag was removed by commit 75ff0feaa275 with the finding that the reasoning was “sort of dubious.” It only ever applied to the domain flag; nohz and managed_irq have no cpuset equivalent at all, which is why the upstream reference configuration still passes isolcpus=managed_irq,7. See the timeline above.
• “The residual 1 Hz tick was removed in the 6.x series.” No. sched_tick_remote() is present and self-requeueing at HZ in v6.12, v6.18 and v7.2 (checked 2026-09-04), and Documentation/admin-guide/cpu-isolation.rst lists it as a standing tradeoff: “Housekeeping CPUs must run a 1Hz residual remote scheduler tick on behalf of the isolated CPUs.”
• “An isolated CPU should be fine making syscalls, it just won’t get the tick.” No — kernel entry and exit are more expensive on a nohz_full CPU, because RCU extended-quiescent-state bookkeeping uses fully ordered read-modify-write operations and CPU-time accounting has to happen on the boundary rather than from the tick. “No call to the kernel from isolated CPUs” is listed upstream as a constraint of the feature.
• “SCHED_DEADLINE is the most deterministic class, so use it on the isolated core.” It is the one class that keeps the tick on unconditionally — sched_can_stop_tick() returns false if rq->dl.dl_nr_running is nonzero at all, even for a single task. SCHED_FIFO never needs the tick.
• “The nohz_full CPU is the one doing the 1 Hz tick work.” No, and this is the useful direction of the confusion: the remote tick runs on a housekeeping CPU via system_unbound_wq and reaches into the isolated run queue under its lock. Unless you have failed to narrow the unbound workqueue mask, in which case it might genuinely be dispatched onto the isolated CPU — which is what makes /sys/devices/virtual/workqueue/cpumask a required step and not a refinement.

Alternatives and When to Choose Them

• Cpuset isolated partitions (the modern, non-deprecated path). Under cgroup v2 you can set a cpuset’s cpuset.cpus.partition to isolated, which performs domain isolation dynamically and reversibly — the kernel even folds these into its cpu_is_isolated() check via cpuset_cpu_is_isolated() (isolation.h, v6.12). This is the recommended replacement for isolcpus=domain and is covered in Cpusets and CPU Partitioning. The boot parameters remain the way to do tick and RCU isolation, which cpusets do not control.
• Affinity-only (CPU Affinity and sched_setaffinity). Pinning a thread and the rest of the system to disjoint CPU masks gets you most of the load-isolation benefit without a reboot, but it does not stop the tick or offload RCU. Use it when you cannot reboot or do not need tickless operation.
• PREEMPT_RT. Orthogonal: PREEMPT_RT bounds worst-case preemption latency kernel-wide; isolation removes jitter sources from specific CPUs. Hard-real-time setups use both.

Production Notes — Who Actually Runs This, and With What

The upstream documentation names one user by name: “high bandwidth network processing that can’t afford losing a single packet or very low latency network processing. Typically those use cases involve DPDK, bypassing the kernel networking stack and performing direct access to the networking device from userspace” (cpu-isolation.rst, v7.2). In practice three families of deployment dominate, and what distinguishes them is less the kernel configuration — which converges — than what they are willing to give up.

	Busy-poll packet processing (DPDK)	Hard real-time control / low-latency trading	Telco NFV (CNF workloads on Kubernetes)
Thread model	One poll-mode driver thread per core, infinite loop, never blocks, never syscalls on the fast path	One SCHED_FIFO thread per control loop, wakes on a hardware or timer event	Mixed: DPDK-style data plane containers plus ordinary control-plane pods
Scheduling class	SCHED_OTHER is usually sufficient — there is nothing to compete with	SCHED_FIFO (never SCHED_RR, never SCHED_DEADLINE, both keep the tick)	SCHED_FIFO for the data plane, SCHED_OTHER for everything else
Domain isolation	isolcpus=domain — the machine has one job for life	isolcpus=domain or an isolated cpuset partition	Cpuset isolated partitions, because the isolated set is per-tenant and changes as pods are scheduled
Tick isolation	nohz_full= on every data-plane core	nohz_full= on the control cores	nohz_full= on the pool the data plane draws from — fixed at boot, which constrains how the pool can be resized
RCU	Implied by nohz_full; rcuo* kthreads pinned to housekeeping	Same, often plus rcu_nocb_poll to trade energy for the removed wakeup	Same
IRQs	irqaffinity=<housekeeping> plus isolcpus=managed_irq; NIC queues bound to the poll cores deliberately, since the driver polls rather than takes interrupts	irqaffinity=<housekeeping>; no I/O at all from the isolated core	Both, per workload class
Kernel	Stock; the fast path never enters the kernel so PREEMPT_RT buys little	PREEMPT_RT, because the thread does enter the kernel and needs bounded preemption latency there	Usually PREEMPT_RT for the RT profile
Memory	Hugepages, mlockall(), NIC buffers pre-allocated	mlockall(MCL_CURRENT | MCL_FUTURE), pre-faulted stacks	Hugepages, mlockall()
Also disabled	SMT (nosmt), deep C-states, cpufreq scaling governors	SMT, C-states, and frequently the machine is a dedicated appliance	SMT per node profile

Three observations from that grid are worth stating explicitly, because they are the ones people get wrong when copying a configuration between the columns.

PREEMPT_RT and isolation solve different problems and are not substitutes. Isolation removes jitter sources from specific CPUs; PREEMPT_RT bounds worst-case preemption latency inside the kernel, machine-wide. DPDK gets little from PREEMPT_RT because its fast path never enters the kernel — there is no kernel latency to bound. A control loop that issues a syscall per iteration needs both. Note also that CONFIG_NUMA_BALANCING explicitly depends on … !PREEMPT_RT, which is the kind of interaction that only shows up when you combine profiles.

SCHED_FIFO, not SCHED_RR or SCHED_DEADLINE. This falls straight out of sched_can_stop_tick(): FIFO is the only real-time class that can be present in any number without keeping the tick, because it has no timeslice to expire. Choosing SCHED_DEADLINE for “more determinism” silently reverts the CPU to fully ticking.

The NFV column is the reason cpuset partitions exist. A boot parameter cannot express “this set of cores is isolated for as long as this pod is scheduled here.” Kubernetes CPU-manager static policy plus a cgroup v2 isolated partition can; isolcpus=domain cannot, and this — not any defect in the parameter — is the whole substance of the cpuset recommendation. Conversely the tick and RCU halves of the configuration remain boot-time and machine-wide, so an NFV node must decide its nohz_full pool before it knows what will run on it. That asymmetry is the practical limit on dynamic CPU isolation today.

Finally, a caution about copying command lines: the folklore recipe isolcpus=domain,managed_irq,2-7 nohz_full=2-7 rcu_nocbs=2-7 and the upstream recipe nohz_full=7 irqaffinity=0-6 isolcpus=managed_irq,7 nosmt are not the same configuration. The upstream one deliberately does domain isolation at runtime, adds a default IRQ affinity mask the folklore version omits entirely, and disables SMT. If you take only one thing from this section, take irqaffinity= and nosmt — they are the two most commonly missing pieces.

See Also

• Housekeeping CPUs and Tickless Isolation — the kernel machinery (HK_TYPE_*, tick_nohz_full_mask, the timekeeping CPU, the residual-tick implementation) that this note’s knobs drive
• Cpusets and CPU Partitioning — the cgroup-based, runtime-reversible replacement for isolcpus=domain
• CPU Affinity and sched_setaffinity — pinning your thread onto an isolated CPU
• Scheduling Domains and CPU Topology — the load-balancing domains that isolcpus=domain removes a CPU from
• Scheduler Load Balancing — the balancer that isolation switches off for a CPU
• The PREEMPT_RT Real-Time Kernel — complementary worst-case-latency bounding
• Linux Process Scheduling MOC — parent map (section F, Affinity, Isolation, and Partitioning)
• The EEVDF Scheduler — what the fair scheduler does with the one task left on an isolated run queue; owns eligibility, lag, virtual deadlines, base_slice, delayed dequeue and the 6.12 fair_server. The two notes meet at sched_can_stop_tick()
• Linux Time and Timers MOC — the periodic tick and nohz machinery this isolation rests on