Context Switch Mechanics: switch_to and switch_mm

When the scheduler has decided that task next should run in place of task prev, the abstract decision must become a physical reality: the CPU has to start executing next’s instructions on next’s kernel stack, with next’s address space mapped and next’s register state restored. That transformation is the context switch, and on Linux/x86-64 it has two distinct halves bridged by one function, context_switch(). First the address space is switched — switch_mm_irqs_off() reloads the page-table base register (CR3) so virtual addresses now resolve through next’s page tables. Then the CPU register and stack state is switched — switch_to() → __switch_to_asm saves prev’s callee-saved registers, swaps the kernel stack pointer (RSP), and __switch_to finishes the per-task hardware state (segments, TLS, FPU bookkeeping). Critically, context_switch() runs holding rq->lock with interrupts disabled, and the matching finish_task_switch() runs on next’s stack to release that lock and clean up after prev (kernel/sched/core.c v6.12). This note traces the path as of Linux 6.12 LTS, x86-64.

All code here is 6.12 LTS, x86-64, and the version pin is deliberate rather than incidental. As of 2026-09-11 kernel.org/releases.json — the machine-readable ground truth, not the human-facing releases page — reports mainline 7.3-rc2, latest stable 7.2.5, and five longterm branches: 6.18.51, 6.12.109, 6.6.156, 6.1.187, 5.15.220. So “6.12 is the LTS” is no longer the whole truth; 6.18 is longterm too. 6.12 is this vault’s pin because it is the branch the surrounding notes were written against, and because symbol names in this area churn: the per-CPU hot-data names (pcpu_hot.current_task, pcpu_hot.top_of_stack), the inactive_task_frame layout, and the mmgrab_lazy_tlb() family have all changed across recent releases. Read the code at the tag you are debugging; nothing below is safe to assume for another kernel.

Everything measured in this note — switch latencies, the thread-versus-process delta, the core-clock calibration — was run on the host this note was written on: an AMD Ryzen AI MAX+ 395 (Zen 5, 16 cores / 32 threads, 2 CCXs, cpuinfo_max_freq 5,187.5 MHz), running kernel 7.1.13-200.fc44.x86_64. Where the running kernel’s behaviour differs from the 6.12 source being quoted, that is called out.

Mental Model

A context switch is a handoff of a single CPU between two tasks, and the trickiest thing about it is that the function performing the switch returns into a different task than the one that called it. The kernel address space is shared across all tasks, so kernel code keeps running across the boundary — but the user mapping, the stack, and the registers all change at the swap point.

flowchart TB
  SCHED["__schedule() picks next,<br/>holds rq->lock, IRQs off"] --> CS["context_switch(rq, prev, next)"]
  CS --> MM{"next->mm == NULL?<br/>(kernel thread?)"}
  MM -->|"yes: lazy TLB"| LAZY["borrow prev->active_mm,<br/>no CR3 reload"]
  MM -->|"no: user task"| SWMM["switch_mm_irqs_off():<br/>load CR3 / PCID,<br/>switch address space"]
  LAZY --> SW["switch_to(prev, next, prev)"]
  SWMM --> SW
  SW --> ASM["__switch_to_asm:<br/>push callee-saved regs of prev,<br/>swap RSP (TASK_threadsp),<br/>pop next's callee-saved regs"]
  ASM --> C["__switch_to (C):<br/>segments, TLS, FPU bookkeeping,<br/>set current/top_of_stack to next"]
  C --> RESUME["execution now on next's stack<br/>as 'next'; returns into the schedule()<br/>next made long ago"]
  RESUME --> FIN["finish_task_switch(prev):<br/>recompute this_rq(), drop rq->lock,<br/>mmdrop prev_mm, reap if prev died"]

The two-phase switch. What it shows: the address-space swap (switch_mm_irqs_off, skipped for kernel threads via the lazy-TLB shortcut) precedes the register/stack swap (switch_to → __switch_to_asm → __switch_to); after the stack swap the CPU is running as next, and finish_task_switch() cleans up on next’s behalf. The insight to take: the line switch_to(prev, next, prev) is where one task “becomes” another — the same C statement that runs as prev returns as next (much later, when next is itself switched away from and back).

The boundary readers actually find hard

The flowchart above hides the one thing that makes context switching genuinely confusing to read: a single mov instruction changes which task the code belongs to, and no C-level construct marks it. Everything before it is prev executing; everything after it is next executing; and the two halves are separated in time by an unbounded interval — microseconds, or minutes, or never, if prev is exiting. Drawn along a time axis with two tasks as separate lifelines:

sequenceDiagram
  participant A as task A (prev)<br/>kernel stack A
  participant CPU as the CPU
  participant B as task B (next)<br/>kernel stack B
  Note over A,CPU: A is running. A calls schedule().
  A->>CPU: __schedule() picks B, takes rq->lock, IRQs off
  A->>CPU: context_switch(rq, A, B)
  A->>CPU: switch_mm_irqs_off() — CR3 now points at B's page tables
  A->>CPU: __switch_to_asm — push A's callee-saved regs onto stack A
  A->>CPU: movq %rsp, TASK_threadsp(%rdi)   (park A's RSP in A->thread.sp)
  rect rgb(240, 225, 225)
  Note over CPU: movq TASK_threadsp(%rsi), %rsp<br/>THE BOUNDARY — one instruction.<br/>Before it the CPU is A. After it the CPU is B.
  end
  CPU->>B: pop B's callee-saved regs off stack B
  CPU->>B: jmp __switch_to — segments, FSBASE, current, sp0
  CPU->>B: ret into the schedule() B called LONG ago
  B->>B: finish_task_switch(A) — drop rq->lock, clear A->on_cpu, reap A if dead
  Note over B: B resumes wherever it blocked.<br/>Its local variable `prev` was overwritten<br/>by switch_to's third argument with<br/>whoever actually ran before this resume.
  Note over A: A's stack is frozen mid-__switch_to_asm.<br/>It will be thawed by whichever task<br/>switches to A next — possibly on<br/>a different CPU, possibly much later.

The switch as two independent lifelines joined by one instruction. What it shows: movq TASK_threadsp(%rsi), %rsp is the seam — the same function is entered by A and left by B, and the “return” of __switch_to_asm happens in a different task, on a different stack, at a different time, than its call. The insight to take: this is why the switch cannot be written in portable C and why switch_to needs a third argument. There is no C-level notion of “return into somebody else”, so the mechanism has to be assembly, and the identity of “who ran before me” has to be smuggled back in a register.

The same idea, stated as a set of invariants that hold on each side of the seam:

	Before movq TASK_threadsp(%rsi), %rsp	After it
%rsp points into	prev’s kernel stack	next’s kernel stack
current (pcpu_hot.current_task)	still prev	still prev — not updated until __switch_to
CR3	already next’s (phase one ran first)	next’s
Callee-saved registers hold	prev’s values, just pushed to prev’s stack	next’s values, about to be popped
rq->lock	held, acquired by prev	held, will be released by next
Interrupts	disabled	disabled
A ret would go to	prev’s caller	wherever next last blocked

The row that trips people is current: for a short window the CPU is executing on next’s stack while current still says prev. That window is entirely inside __switch_to_asm and __switch_to, runs with interrupts off, and is closed by raw_cpu_write(pcpu_hot.current_task, next_p). Kernel code that runs in that window (the FILL_RETURN_BUFFER macro, the canary reload) must not use current.

The Entry: context_switch()

The whole switch is orchestrated by context_switch(), called from the core scheduler (see The Core Scheduler and __schedule). It is entered with rq->lock held and interrupts disabled:

static __always_inline struct rq *
context_switch(struct rq *rq, struct task_struct *prev,
	       struct task_struct *next, struct rq_flags *rf)
{
	prepare_task_switch(rq, prev, next);
	arch_start_context_switch(prev);
	/* ... address-space switch (below) ... */
	prepare_lock_switch(rq, next, rf);
	switch_to(prev, next, prev);   /* register + stack swap */
	barrier();
	return finish_task_switch(prev);
}

From core.c v6.12. prepare_task_switch() runs sched-out notifiers, perf accounting, and rseq bookkeeping. prepare_lock_switch() records that the lock is being handed to next (so lockdep and the on_cpu state stay consistent). Then switch_to() does the irreversible swap, barrier() forbids the compiler from moving memory accesses across it, and finish_task_switch() — running now as next — tidies up.

Phase One: switch_mm and the Address-Space Swap

The address-space decision turns on whether next is a user task or a kernel thread, encoded in next->mm:

	if (!next->mm) {                                // to kernel
		enter_lazy_tlb(prev->active_mm, next);
		next->active_mm = prev->active_mm;
		if (prev->mm)                           // from user
			mmgrab_lazy_tlb(prev->active_mm);
		else
			prev->active_mm = NULL;
	} else {                                        // to user
		membarrier_switch_mm(rq, prev->active_mm, next->mm);
		switch_mm_irqs_off(prev->active_mm, next->mm, next);
		lru_gen_use_mm(next->mm);
		if (!prev->mm) {                        // from kernel
			rq->prev_mm = prev->active_mm;
			prev->active_mm = NULL;
		}
	}

From core.c v6.12. Two fields drive this: mm is the task’s real address space (NULL for kernel threads, which have no user mappings) and active_mm is the address space currently loaded on the CPU. Per the kernel’s active_mm documentation, the foundational rule is: “for a process with a real address space (ie tsk→mm is non-NULL) the active_mm obviously always has to be the same as the real one.”

Switching to a kernel thread (!next->mm) — the lazy-TLB shortcut. A kernel thread touches only kernel memory, which is mapped identically in every address space. Reloading CR3 would needlessly flush user TLB entries for no benefit, so the kernel borrows the outgoing task’s address space: next->active_mm = prev->active_mm, and no switch_mm is called at all. The mmgrab_lazy_tlb() takes a lightweight reference (on mm_count, not mm_users) so the borrowed mm_struct cannot be freed while the kernel thread “lazily” holds it. This is the lazy TLB mechanism — covered in depth in The mm_struct and Process Address Space (the active_mm borrow) and The Translation Lookaside Buffer and TLB Shootdowns (why skipping the flush is safe and how shootdowns reach lazy-TLB CPUs).

Switching to a user task (next->mm non-NULL). Here switch_mm_irqs_off(prev->active_mm, next->mm, next) does the real address-space switch: on x86-64 it loads next->mm’s top-level page-table physical address into CR3. With PCID (Process-Context Identifiers) the write tags the new address space rather than flushing the whole TLB, so entries for both old and new address spaces can coexist and a later switch back is cheap. The mechanics of CR3/PCID, lazy-TLB tracking, and cross-CPU invalidation belong to The Translation Lookaside Buffer and TLB Shootdowns and are not re-derived here. If the previous task was a kernel thread (!prev->mm), the borrowed active_mm is no longer needed and is stashed in rq->prev_mm to be dropped later, outside the run-queue lock, in finish_task_switch().

The membarrier_switch_mm() call and the comment around it encode an ordering guarantee for the membarrier() syscall: there must be a full memory barrier between updating rq->curr and returning to userspace; switch_mm() supplies it on the switch path, and finish_task_switch()’s mmdrop supplies it when prev->active_mm == next->mm. A third call, switch_mm_cid(rq, prev, next), sits immediately after the if/else and updates the per-mm concurrency ID used by rseq; its comment (“switch_mm_cid() requires the memory barriers above”) makes explicit that it is a consumer of the barrier switch_mm() just provided, not an independent step.

Inside switch_mm_irqs_off: three exits, not one

Calling switch_mm_irqs_off() is not the same as switching the address space. The x86 implementation has three distinct outcomes, and only one of them writes CR3. The first surprise is its signature (arch/x86/mm/tlb.c, v6.12):

void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next,
			struct task_struct *tsk)
{
	struct mm_struct *prev     = this_cpu_read(cpu_tlbstate.loaded_mm);
	u16               prev_asid = this_cpu_read(cpu_tlbstate.loaded_mm_asid);
	bool              was_lazy  = this_cpu_read(cpu_tlbstate_shared.is_lazy);

The first parameter is literally named unused. The scheduler passes prev->active_mm, but x86 ignores it and reads the authoritative answer — which mm is actually loaded in CR3 on this CPU — out of the per-CPU cpu_tlbstate. That is not pedantry: the scheduler’s idea of prev->active_mm and the CPU’s idea of the loaded mm can legitimately differ after lazy-TLB borrowing, and only the per-CPU state is authoritative about the hardware.

The three exits:

Exit 1 — same mm, not lazy: return without touching CR3 at all. This is the two-threads-of-one-process case, and the code is explicit about it:

	if (prev == next) {
		/* Not actually switching mm's */
		...
		/*
		 * If the CPU is not in lazy TLB mode, we are just switching
		 * from one thread in a process to another thread in the same
		 * process. No TLB flush required.
		 */
		if (!was_lazy)
			return;

Exit 2 — same mm, but the CPU was lazy. While the CPU sat in lazy-TLB mode running a kernel thread, some other CPU may have invalidated mappings in this mm and skipped this CPU’s IPI precisely because it was lazy. So on the way out of lazy mode the code re-reads the mm’s TLB generation counter and only falls through to a flush if it is behind:

		smp_mb();
		next_tlb_gen = atomic64_read(&next->context.tlb_gen);
		if (this_cpu_read(cpu_tlbstate.ctxs[prev_asid].tlb_gen) == next_tlb_gen)
			return;
		/* TLB contents went out of date while we were in lazy mode. */
		new_asid = prev_asid;
		need_flush = true;
	}

Exit 3 — a genuinely different mm. Only here does the CPU leave mm_cpumask(prev), join mm_cpumask(next), pick an ASID, and write CR3. It also runs cond_mitigation(tsk) — the conditional indirect-branch-predictor barrier discussed below.

flowchart TB
  E["context_switch() reaches the mm decision"] --> K{"next->mm == NULL?"}
  K -->|"yes — kernel thread"| LAZY["LAZY TLB<br/>next->active_mm = prev->active_mm<br/>mmgrab_lazy_tlb()<br/><b>switch_mm_irqs_off is never called</b>"]
  K -->|"no — user task"| CALL["call switch_mm_irqs_off()"]
  CALL --> P{"loaded_mm == next?<br/>(read from per-CPU cpu_tlbstate,<br/>NOT from the 'unused' argument)"}
  P -->|"no"| DIFF["different address space:<br/>cond_mitigation() — maybe IBPB<br/>leave mm_cpumask(prev), join mm_cpumask(next)<br/>choose_new_asid()<br/><b>write CR3</b>"]
  P -->|"yes"| L{"was this CPU lazy?"}
  L -->|"no"| RET0["<b>return immediately</b><br/>no CR3 write, no flush<br/>(thread -> thread, same process)"]
  L -->|"yes"| G{"cpu_tlbstate.ctxs[asid].tlb_gen<br/>== next->context.tlb_gen ?"}
  G -->|"yes, TLB still current"| RET1["<b>return</b> — reuse the loaded ASID<br/>no flush needed"]
  G -->|"no, went stale while lazy"| FLUSH["reuse same ASID but<br/><b>write CR3 with flush</b>"]
  DIFF --> ASID{"choose_new_asid()"}
  ASID -->|"X86_FEATURE_PCID absent"| A0["ASID 0, need_flush = true<br/><i>always a flushing CR3 write</i>"]
  ASID -->|"mm already owns an ASID<br/>on this CPU and tlb_gen is current"| AKEEP["reuse ASID, <b>no flush</b><br/>— the PCID win"]
  ASID -->|"no slot, or stale"| ANEW["allocate next ASID slot<br/>(wraps at TLB_NR_DYN_ASIDS = 6)<br/>need_flush = true"]

The address-space decision as the kernel actually makes it. What it shows: four of the seven leaves do no CR3 write — the kernel-thread shortcut, the same-process-thread fast return, the lazy-but-current return, and the PCID ASID-reuse path. The insight to take: “a context switch reloads the page tables” is wrong more often than it is right. The expensive path is reached only when the destination address space genuinely differs and the CPU cannot prove its cached translations for that address space are still valid.

ASIDs, PCID, and the number 6

x86’s Process-Context Identifier (PCID) feature lets CR3 carry a 12-bit tag so TLB entries for several address spaces coexist; without it, every CR3 write flushes all non-global entries. Linux does not use the hardware’s full 4,096-value PCID space. It maintains a tiny per-CPU array of software ASIDs mapped onto kernel PCIDs, sized by a single constant (arch/x86/include/asm/tlbflush.h, v6.12):

#define TLB_NR_DYN_ASIDS	6
...
	struct tlb_context ctxs[TLB_NR_DYN_ASIDS];

Six. choose_new_asid() linearly scans those six slots for one already holding next’s context id; on a hit it flushes only if the recorded tlb_gen is behind the mm’s current generation, and on a miss it round-robins a slot and forces a flush. The comment block above kern_pcid() records the mapping: software ASIDs occupy [0, TLB_NR_DYN_ASIDS-1], kernel PCIDs [1, TLB_NR_DYN_ASIDS], and — when Kernel Page-Table Isolation is in play — user PCIDs [2048+1, 2048+TLB_NR_DYN_ASIDS], the 2048 bit being the PTI user/kernel switch bit.

The practical consequence is that PCID only helps when a CPU cycles among at most six address spaces. A CPU round-robining ten processes evicts ASID slots faster than it reuses them and flushes on nearly every switch anyway.

Situation	CR3 written?	TLB flushed?	Notes
→ kernel thread (lazy TLB)	no	no	switch_mm_irqs_off() not even called
→ another thread of the same process, CPU not lazy	no	no	early return in switch_mm_irqs_off()
→ same mm, CPU was lazy, tlb_gen current	no	no	the generation check pays for itself here
→ same mm, CPU was lazy, tlb_gen stale	yes	yes	same ASID, flushing write
→ different mm, PCID present, ASID slot still valid	yes	no	the PCID win: tagged, non-flushing CR3 write
→ different mm, PCID present, no slot / stale	yes	yes	ASID reallocated round-robin among 6
→ different mm, no PCID	yes	yes	choose_new_asid() forces ASID 0 + flush, always

Uncertain

Verify: whether this particular host actually gets the PCID path. Reason: /proc/cpuinfo on this AMD Zen 5 machine lists invpcid but not pcid in its flags, and choose_new_asid() begins if (!static_cpu_has(X86_FEATURE_PCID)) { *new_asid = 0; *need_flush = true; return; } — which would mean every cross-process switch here takes a flushing CR3 write. That reading may be wrong (the flag could be masked or reported differently on this part, or the running 7.1 kernel may differ from the 6.12 source quoted). To resolve: check X86_FEATURE_PCID directly via /sys/kernel/debug/x86/ or a kernel tracepoint (trace_tlb_flush distinguishes TLB_FLUSH_ON_TASK_SWITCH with and without TLB_FLUSH_ALL), which needs root and debugfs — not available in this environment. Treat the measured cross-process delta below as an upper bound that may include a full flush. uncertain

The mitigation tax

Switching between different user address spaces also triggers branch-predictor hygiene that switching between threads of one process does not. In switch_mm_irqs_off()’s “different mm” arm, cond_mitigation(tsk) may issue an Indirect Branch Prediction Barrier:

	/*
	 * Avoid user/user BTB poisoning by flushing the branch predictor
	 * when switching between processes. This stops one process from
	 * doing Spectre-v2 attacks on another.
	 *
	 * Both, the conditional and the always IBPB mode use the mm
	 * pointer to avoid the IBPB when switching between tasks of the
	 * same process.
	 */

Note the last clause: the mitigation is deliberately keyed on the mm pointer so that thread-to-thread switches inside one process skip it entirely. That is a second, independent reason process switches cost more than thread switches, on top of the CR3 write — and it is invisible in the scheduler source, because it lives inside the memory-management code. On this host /sys/devices/system/cpu/vulnerabilities/spectre_v2 reports Mitigation: Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on, so the conditional path is the one in force. Separately, __switch_to_asm runs FILL_RETURN_BUFFER on every switch where X86_FEATURE_RSB_CTXSW is set, overwriting the Return Stack Buffer — that one is not conditional on the address space and is paid by thread switches too.

One mitigation this machine does not pay: /sys/devices/system/cpu/vulnerabilities/meltdown reports Not affected, so Kernel Page-Table Isolation is off and there is no PTI CR3 toggle on syscall entry and exit. On a PTI-enabled Intel machine the same benchmarks below would be measurably slower, and the CR3 writes would be roughly doubled.

Phase Two: switch_to and the Register/Stack Swap

switch_to is a macro, and its three-argument shape is the single most important detail of the whole mechanism:

#define switch_to(prev, next, last)					\
do {									\
	((last) = __switch_to_asm((prev), (next)));			\
} while (0)

From arch/x86/include/asm/switch_to.h v6.12. Why three arguments and not two? When a task is switched out, its kernel stack — including the local variables of the very schedule() call that switched it away — is frozen in place. When the scheduler later switches it back in, execution resumes exactly there, with those frozen locals restored. But the world has moved on: the task that ran immediately before this resume is almost never the same task this one switched away from. The resumed task needs to know who actually ran last so finish_task_switch() can clean up after the right prev. That is what last carries: __switch_to_asm returns (in %rax) the task that was running just before the resume, and the macro assigns it to last. In context_switch() the call is switch_to(prev, next, prev) — so on resumption the local prev is overwritten with the genuine previous task, and finish_task_switch(prev) then operates on the correct one.

The architecture-specific swap is __switch_to_asm, hand-written assembly:

SYM_FUNC_START(__switch_to_asm)
	/* Save callee-saved registers (order matches inactive_task_frame) */
	pushq	%rbp
	pushq	%rbx
	pushq	%r12
	pushq	%r13
	pushq	%r14
	pushq	%r15
 
	/* switch stack */
	movq	%rsp, TASK_threadsp(%rdi)   /* save prev's RSP into prev->thread.sp */
	movq	TASK_threadsp(%rsi), %rsp   /* load next's RSP from next->thread.sp */
 
#ifdef CONFIG_STACKPROTECTOR
	movq	TASK_stack_canary(%rsi), %rbx
	movq	%rbx, PER_CPU_VAR(fixed_percpu_data + FIXED_stack_canary)
#endif
	FILL_RETURN_BUFFER %r12, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_CTXSW
 
	/* restore next's callee-saved registers */
	popq	%r15
	popq	%r14
	popq	%r13
	popq	%r12
	popq	%rbx
	popq	%rbp
 
	jmp	__switch_to
SYM_FUNC_END(__switch_to_asm)

From arch/x86/entry/entry_64.S v6.12. %rdi is prev, %rsi is next (System V calling convention). The function:

1. Pushes prev’s callee-saved registers (rbp, rbx, r12–r15) onto prev’s kernel stack. Only callee-saved registers are saved here; caller-saved ones were already spilled by the C compiler around the switch_to call. The push order exactly mirrors struct inactive_task_frame, which is what an inactive task’s thread.sp points at.
2. Swaps the kernel stack — movq %rsp, TASK_threadsp(%rdi) parks prev’s stack pointer in prev->thread.sp; movq TASK_threadsp(%rsi), %rsp loads next’s. This single instruction is the moment the CPU stops being prev and starts being next — from here, every push/pop and ret walks next’s stack.
3. Reloads the stack-protector canary for next (per-task canary) and runs FILL_RETURN_BUFFER to overwrite the Return Stack Buffer, a Spectre-class mitigation applied on context switch.
4. Pops next’s callee-saved registers — but these are the values that were pushed when next was last switched out, now restored from next’s stack.
5. jmp __switch_to — a tail call into the C continuation, which will eventually ret back to wherever next was when it last called schedule().

The C half, __switch_to, handles the remaining per-task hardware state:

__visible __notrace_funcgraph struct task_struct *
__switch_to(struct task_struct *prev_p, struct task_struct *next_p)
{
	/* ... */
	if (!test_tsk_thread_flag(prev_p, TIF_NEED_FPU_LOAD))
		switch_fpu_prepare(prev_p, cpu);
	save_fsgs(prev_p);
	load_TLS(next, cpu);
	/* ES/DS, FS/GS base, PKRU ... */
	raw_cpu_write(pcpu_hot.current_task, next_p);
	raw_cpu_write(pcpu_hot.top_of_stack, task_top_of_stack(next_p));
	switch_fpu_finish(next_p);
	update_task_stack(next_p);   /* reload sp0 (entry-stack pointer) */
	/* ... */
	return prev_p;
}

From arch/x86/kernel/process_64.c v6.12. It saves prev’s FS/GS bases and segment selectors, loads next’s Thread-Local Storage GDT entries (load_TLS), restores FS/GS bases and the PKRU (memory-protection-keys) register, updates the per-CPU current_task and top_of_stack pointers to next, and reloads sp0 (the kernel entry-stack pointer the CPU uses on the next user→kernel transition). The returned prev_p becomes __switch_to_asm’s return value in %rax — feeding the last mechanism described above.

FPU is deferred, not eagerly reloaded. A common misconception is that __switch_to saves and restores the full floating-point/SIMD register file. It does not. switch_fpu_prepare() saves prev’s FPU state to memory only if prev doesn’t already have the TIF_NEED_FPU_LOAD flag set, and switch_fpu_finish() merely sets up so that next’s FPU registers are reloaded lazily on return to userspace, not here. This avoids touching the (large) AVX/AVX-512 state for tasks that never use it before blocking again.

What the switch saves, and what it deliberately does not

The list of things a context switch skips is longer and more interesting than the list of things it saves. Everything below is from __switch_to_asm and __switch_to as of v6.12:

State	Saved/restored by the switch?	Where, and why not
Callee-saved GPRs (rbx, rbp, r12–r15)	yes, by __switch_to_asm	pushed onto prev’s stack in struct inactive_task_frame order
Caller-saved GPRs (rax, rcx, rdx, rsi, rdi, r8–r11)	no	already spilled by the C compiler around the switch_to call — the switch is a function call, and the ABI did the work
Kernel stack pointer %rsp	yes	via prev->thread.sp / next->thread.sp (TASK_threadsp)
RIP / return address	implicitly	it is simply the return address sitting on each task’s own frozen stack
RFLAGS	no (on x86-64)	inactive_task_frame carries flags only in the 32-bit arm of the #ifdef
User pt_regs (all user registers)	no	saved on kernel entry, restored on kernel exit — a different mechanism entirely
CR3 / page tables	phase one, conditionally	see the three-exit analysis above
FS base — the userspace thread pointer	yes	save_fsgs() / x86_fsgsbase_load(); see the next subsection
GS base (inactive, i.e. user GS)	yes	__rdgsbase_inactive() / __wrgsbase_inactive()
DS, ES selectors	yes, but conditionally	`if (unlikely(next→es
CS, SS	no	“saved and restored as part of pt_regs”, per the source comment
GDT TLS entries (the three set_thread_area slots)	yes	load_TLS(next, cpu) — this is the legacy 32-bit TLS mechanism, not __thread
PKRU (memory protection keys)	yes, conditionally	x86_pkru_load() skips wrpkru when prev->pkru == next->pkru, because “PKRU writes are slightly expensive”
FPU / SSE / AVX / AVX-512 register file	deferred	saved only if not already saved; reloaded on return to userspace, not here
Kernel stack canary	yes	movq TASK_stack_canary(%rsi), %rbx into the per-CPU fixed_percpu_data — a per-task value
Return Stack Buffer	overwritten	FILL_RETURN_BUFFER — not “saved”, destroyed on purpose as a Spectre mitigation
Branch predictor (BTB)	conditionally flushed	cond_mitigation() in phase one, and only across different mms
sp0 (entry-stack pointer in the TSS)	yes	update_task_stack(next_p), so the next user→kernel transition lands on next’s stack
current / top_of_stack per-CPU pointers	yes	raw_cpu_write(pcpu_hot.current_task, next_p) and pcpu_hot.top_of_stack
Debug registers, LDT, IO bitmap	conditionally	switch_to_extra() handles the uncommon cases; switch_ldt() runs from switch_mm_irqs_off() and only when the mm changed

The pattern is consistent: anything the entry/exit path already handles is not handled again, and anything whose write is expensive is guarded by a comparison first. x86_pkru_load() is the clearest miniature of that philosophy — the kernel would rather read a register and branch than unconditionally execute one wrpkru.

The FSBASE bridge: __switch_to restores the userspace thread pointer

One row of that table deserves its own treatment, because it is where this note meets TLS Models and Access. The FS segment base is not a scheduler implementation detail — on x86-64 it is the userspace thread pointer, the value that every %fs:-prefixed instruction in every thread-local variable access implicitly adds. Saving it (process_64.c, v6.12):

static __always_inline void save_fsgs(struct task_struct *task)
{
	savesegment(fs, task->thread.fsindex);
	savesegment(gs, task->thread.gsindex);
	if (static_cpu_has(X86_FEATURE_FSGSBASE)) {
		/*
		 * If FSGSBASE is enabled, we can't make any useful guesses
		 * about the base, and user code expects us to save the current
		 * value.  Fortunately, reading the base directly is efficient.
		 */
		task->thread.fsbase = rdfsbase();
		task->thread.gsbase = __rdgsbase_inactive();
	} else {
		save_base_legacy(task, task->thread.fsindex, FS);
		save_base_legacy(task, task->thread.gsindex, GS);
	}
}

and restoring it:

static __always_inline void x86_fsgsbase_load(struct thread_struct *prev,
					      struct thread_struct *next)
{
	if (static_cpu_has(X86_FEATURE_FSGSBASE)) {
		if (unlikely(prev->fsindex || next->fsindex))
			loadseg(FS, next->fsindex);
		if (unlikely(prev->gsindex || next->gsindex))
			loadseg(GS, next->gsindex);
		wrfsbase(next->fsbase);
		__wrgsbase_inactive(next->gsbase);
	} else {
		load_seg_legacy(prev->fsindex, prev->fsbase,
				next->fsindex, next->fsbase, FS);
		load_seg_legacy(prev->gsindex, prev->gsbase,
				next->gsindex, next->gsbase, GS);
	}
}

Follow one thread-local read all the way down. A __thread int x compiled local-exec becomes mov %fs:0xfffffffffffffffc,%eax — a load four bytes below the FS base. Nothing in that instruction names a thread. It resolves to this thread’s copy purely because wrfsbase(next->fsbase) executed a few hundred nanoseconds earlier, inside __switch_to, on this CPU. The compiler’s entire four-model taxonomy — which relocation, which resolver, how many indirections — decides what offset to add to the thread pointer; __switch_to decides what the thread pointer is. Neither half means anything without the other.

Two details of the restore path matter for cost:

• FSGSBASE is a CPU feature, and its absence is expensive. With X86_FEATURE_FSGSBASE the kernel uses the RDFSBASE/WRFSBASE instructions, which the comment above calls “efficient”. Without it, the fallback in load_seg_legacy() reaches for wrmsrl(MSR_FS_BASE, ...) — a model-specific-register write, which is serialising and costs on the order of a hundred cycles. This host advertises fsgsbase, so it takes the fast path.
• The legacy path guesses, and AMD makes guessing unsafe. save_base_legacy() carries a long comment explaining that when the selector is zero it simply assumes the saved value is still correct, because “this branch is very hot (it’s hit twice on almost every context switch between 64-bit programs), and avoiding the RDMSR helps a lot” — while noting that on AMD (with X86_BUG_NULL_SEG) “the segment base could be almost anything”. The optimisation is a deliberate, documented inaccuracy preserved for compatibility: “this matches historical Linux behavior, so it won’t break existing applications.”

A corollary worth stating because it is a common confusion: the kernel stack canary reloaded in __switch_to_asm (TASK_stack_canary) is genuinely per task, whereas the userspace canary read as %fs:0x28 is per process — every thread of a process shares it. Same idea, two different scopes, two different mechanisms; the measurement is in TLS Models and Access.

Phase Three: finish_task_switch()

After switch_to returns, the CPU is executing as next, on next’s stack — and next long ago called schedule(), so its restored locals reflect that call site, possibly on a different CPU than where it last ran. finish_task_switch() reconciles all of this:

static struct rq *finish_task_switch(struct task_struct *prev)
	__releases(rq->lock)
{
	struct rq *rq = this_rq();          /* recompute: prev may be on another CPU */
	struct mm_struct *mm = rq->prev_mm;
	unsigned int prev_state;
	/* ... preempt_count sanity check ... */
	rq->prev_mm = NULL;
	prev_state = READ_ONCE(prev->__state);
	vtime_task_switch(prev);
	finish_task(prev);                  /* clears prev->on_cpu */
	tick_nohz_task_switch();
	finish_lock_switch(rq);             /* drops rq->lock */
	/* ... */
	if (mm) {
		membarrier_mm_sync_core_before_usermode(mm);
		mmdrop_lazy_tlb_sched(mm);      /* drop the borrowed kernel-thread mm */
	}
	if (unlikely(prev_state == TASK_DEAD)) {
		if (prev->sched_class->task_dead)
			prev->sched_class->task_dead(prev);
		put_task_stack(prev);
		put_task_struct_rcu_user(prev);  /* final reference if prev exited */
	}
	return rq;
}

From core.c v6.12. The header comment states the key fact plainly: “The context switch have flipped the stack from under us … ‘prev == current’ is still correct but we need to recalculate this_rq because prev may have moved to another CPU.” So:

1. this_rq() is recomputed rather than reused — the run queue this code runs on may not be the one prev was switched out on.
2. The run-queue lock handoff completes. context_switch() was entered with rq->lock held by prev; finish_task_switch() (running as next) releases it via finish_lock_switch(). This is the lock handoff across the switch: one task acquires the lock, a different task releases it. finish_task() clears prev->on_cpu, the barrier that lets prev be safely picked up by a wakeup on another CPU.
3. The deferred prev_mm is dropped. If phase one stashed a borrowed kernel-thread mm in rq->prev_mm, mmdrop_lazy_tlb_sched() releases that reference now — deliberately outside the run-queue lock, because mmdrop can do real work and “doing it with the lock held can cause deadlocks.”
4. A dead prev is reaped. If prev set TASK_DEAD and called schedule() for the last time, this is where its stack is freed (put_task_stack) and its final task_struct reference is dropped via RCU. A task literally cannot free its own stack while running on it — so the next task does it. (See Process Exit wait and Zombie Reaping for the surrounding lifecycle.)

The finish_task() step deserves a closer look, because it is the point at which prev becomes other CPUs’ business again (kernel/sched/core.c, v6.12):

static inline void finish_task(struct task_struct *prev)
{
#ifdef CONFIG_SMP
	/*
	 * This must be the very last reference to @prev from this CPU. After
	 * p->on_cpu is cleared, the task can be moved to a different CPU. We
	 * must ensure this doesn't happen until the switch is completely
	 * finished.
	 *
	 * In particular, the load of prev->state in finish_task_switch() must
	 * happen before this.
	 *
	 * Pairs with the smp_cond_load_acquire() in try_to_wake_up().
	 */
	smp_store_release(&prev->on_cpu, 0);
#endif
}

That one smp_store_release is a release-fence publication: until it lands, try_to_wake_up() on another CPU spins in smp_cond_load_acquire(&p->on_cpu, !VAL) rather than touching a task whose registers may still be half-saved. It is also why finish_task_switch() reads prev->__state into a local before calling finish_task() — after the release, prev may already be running elsewhere and its state field is no longer stable.

stateDiagram-v2
  direction LR
  [*] --> Running_prev
  Running_prev: prev running<br/>on_cpu = 1
  Running_prev --> Locked: __schedule() takes rq->lock,<br/>disables IRQs
  Locked: rq->lock HELD by prev<br/>IRQs off
  Locked --> Seam: switch_mm (maybe) then __switch_to_asm
  Seam: THE SEAM<br/>stack swapped, current still says prev<br/>rq->lock still held — by nobody in particular
  Seam --> Running_next: __switch_to sets current = next,<br/>ret into next's old schedule()
  Running_next: next executing finish_task_switch(prev)
  Running_next --> Released: finish_lock_switch() drops rq->lock
  Released --> Published: smp_store_release(&prev->on_cpu, 0)
  Published: prev is now claimable by<br/>any CPU's try_to_wake_up()
  Published --> Reaped: if prev->__state == TASK_DEAD
  Reaped: put_task_stack(prev)<br/>put_task_struct_rcu_user(prev)
  Published --> [*]: otherwise prev simply waits
  Reaped --> [*]

The lock and ownership handoff across a switch. What it shows: rq->lock is acquired by one task and released by a different one, and prev is not safe for another CPU to touch until on_cpu is published with a release store — several steps after the stack was swapped. The insight to take: the switch is not “done” when the registers change. There is a tail, executed by the incoming task on the outgoing task’s behalf, and it is that tail — not the register swap — that contains the concurrency subtleties.

Newly Forked Tasks and ret_from_fork

A brand-new task has never run, so it has no frozen schedule() to return into. copy_thread() hand-builds an inactive_task_frame whose ret_addr points at ret_from_fork_asm, so the first time the scheduler switches to the new task, __switch_to_asm’s final ret lands there:

SYM_CODE_START(ret_from_fork_asm)
	/* ... */
	movq	%rax, %rdi		/* prev */
	movq	%rsp, %rsi		/* regs */
	movq	%rbx, %rdx		/* fn  (kernel-thread function, NULL for user) */
	movq	%r12, %rcx		/* fn_arg */
	call	ret_from_fork
	/* ... */
	jmp	swapgs_restore_regs_and_return_to_usermode
SYM_CODE_END(ret_from_fork_asm)

From entry_64.S v6.12. The new task first calls schedule_tail() (which calls finish_task_switch() for it — completing the lock handoff its first time on a CPU), then either runs a kernel-thread function or returns to userspace. The %rbx/%r12 registers carry the kernel-thread function and argument, planted by kthread_frame_init() (switch_to.h, v6.12):

static inline void kthread_frame_init(struct inactive_task_frame *frame,
				      int (*fun)(void *), void *arg)
{
	frame->bx  = (unsigned long)fun;
	frame->r12 = (unsigned long)arg;      /* x86-64 arm of the #ifdef */
}

The trick is entirely one of stack forgery. __switch_to_asm has exactly one way to resume a task — pop six registers and ret — so copy_thread() builds, by hand, a stack that looks as though it were left there by a previous __switch_to_asm, with a ret_addr of its choosing. The layout it must match is declared in the same header:

struct inactive_task_frame {
	unsigned long r15;
	unsigned long r14;
	unsigned long r13;
	unsigned long r12;
	unsigned long bx;
	/* These two fields must be together.  They form a stack frame
	 * header, needed by get_frame_pointer(). */
	unsigned long bp;
	unsigned long ret_addr;
};
 
struct fork_frame {
	struct inactive_task_frame frame;
	struct pt_regs             regs;
};

flowchart TB
  subgraph SW["a task switched out the NORMAL way"]
    direction TB
    N1["higher addresses"] --> N2["... rest of the kernel stack ...<br/>the frozen schedule() frame"]
    N2 --> N3["ret_addr — inside __switch_to_asm's caller<br/>(the schedule() this task was in)"]
    N3 --> N4["bp, bx, r12, r13, r14, r15<br/>pushed by __switch_to_asm"]
    N4 --> N5["thread.sp points here"]
  end
  subgraph FK["a BRAND-NEW task, stack forged by copy_thread()"]
    direction TB
    F1["top of stack"] --> F2["struct pt_regs<br/>(the user register state fork() will return with,<br/>or garbage for a kernel thread)"]
    F2 --> F3["ret_addr = ret_from_fork_asm<br/><b>hand-written, never pushed by anyone</b>"]
    F3 --> F4["bp = 0<br/>bx = kernel-thread fn (or 0)<br/>r12 = fn arg (or 0)<br/>r13, r14, r15 = 0"]
    F4 --> F5["thread.sp points here"]
  end
  SW -.->|"both shapes are<br/>byte-compatible, so<br/>__switch_to_asm cannot<br/>tell them apart"| FK

How a never-run task is made resumable. What it shows: struct fork_frame places a forged inactive_task_frame exactly where __switch_to_asm’s pops and ret expect one, with ret_addr pointing at ret_from_fork_asm instead of at a real return site, and with bx/r12 doubling as smuggled arguments. The insight to take: the kernel does not have a separate “start a new task” path. It has one resume path, and creating a task means building a stack that lies convincingly enough for that path to work.

Two consequences follow from the forgery. First, ret_from_fork_asm receives %rax = the real prev (from __switch_to’s return value), so even a task that has never run participates correctly in the last mechanism. Second, struct fork_frame bundles the forged frame with a struct pt_regs, so after ret_from_fork() finishes the task can fall straight into swapgs_restore_regs_and_return_to_usermode and appear to return from the fork()/clone() syscall that created it — which is exactly what userspace expects.

Measured: What a Context Switch Actually Costs

All of the above is structure. The question a reader eventually wants answered is how much, and the answer turns out to depend far less on the switch itself than on where the two tasks are.

Methodology

A ping-pong benchmark: two peers alternate, each blocking until the other wakes it, so every round trip contains exactly two voluntary context switches. Two IPC mechanisms are used independently, because each carries its own overhead and agreeing results are more convincing than either alone:

• pipe — peer A write()s one byte and blocks in read(); peer B does the mirror image. Two pipes, so neither side can read its own byte.
• futex — a shared word in a MAP_SHARED anonymous mapping, with FUTEX_WAKE/FUTEX_WAIT and acquire/release atomics. Cheaper than a pipe because there is no file-descriptor or pipe-buffer work.

Each is run twice: once with the peer as a thread of the same process (so phase one takes the prev == next early return — no CR3 write, no IBPB) and once with the peer as a forked process (so phase one takes the full different-mm path). The difference between those two numbers, with everything else identical, is the cost of the address-space switch plus its mitigations.

Both peers are pinned with sched_setaffinity(). 2,000 warm-up round trips precede timing; 300,000 timed round trips follow. Timing is clock_gettime(CLOCK_MONOTONIC) around the whole timed loop. A getppid() syscall loop (1,000,000 iterations after 100,000 warm-up) provides a syscall-overhead baseline, so the pure-syscall component can be subtracted mentally.

Separating TSC ticks from core cycles. This is not optional on this machine. The invariant TSC runs at a fixed 2.9943 GHz while the core boosts to about 5.05 GHz; reporting rdtsc deltas as “cycles” under-reports by 1 − 2.9943/5.05 ≈ 41%. The core clock is therefore measured separately inside the benchmark process, immediately before timing, using a dependent-ADD tachometer: a loop of add $1,%reg instructions forming a single one-cycle-latency dependency chain, so the loop retires exactly one ADD per core cycle whatever the frequency; ADDs ÷ elapsed wall time = achieved core clock. Validated against three chain depths and against the kernel’s own scaling_cur_freq:

$ ./clk2 0 ; ./clk2 4 ; ./clk2 16
cpu0   TSC=2.9943 GHz | tach K=1: 5.0639 | K=2: 4.8676 | K=4: 5.0150 GHz | sysfs 5.125 GHz
cpu4   TSC=2.9943 GHz | tach K=1: 4.9869 | K=2: 4.9739 | K=4: 5.0815 GHz | sysfs 5.042 GHz
cpu16  TSC=2.9943 GHz | tach K=1: 4.9051 | K=2: 4.9408 | K=4: 4.9925 GHz | sysfs 4.967 GHz

(That the three chain depths agree is the check that the tachometer is measuring cycles and not something else; an early version of this loop divided the ADD count by the chain depth and produced a nonsense 1.25 GHz for K=4, which the cross-check caught.) Every cycle figure below is ns × core_GHz using a freshly measured core clock, never a TSC delta.

Both peers on the same logical CPU

$ ./ctxsw 300000 4 4
core=5.0584 GHz  cpu_main=4 cpu_peer=4 iters=300000
  getppid() syscall (baseline)           37.0 ns    187.3 core cycles
  pipe  round trip, threads            1520.1 ns   7689.1 core cycles   -> per switch  760.0 ns / 3844.6 cyc
  pipe  round trip, processes          1934.7 ns   9786.3 core cycles   -> per switch  967.3 ns / 4893.2 cyc
  futex round trip, threads            1372.6 ns   6943.3 core cycles   -> per switch  686.3 ns / 3471.6 cyc
  futex round trip, processes          1789.1 ns   9049.9 core cycles   -> per switch  894.5 ns / 4525.0 cyc

Three independent repeats agreed to within about 2% (pipe/threads: 1520.1, 1550.3, 1537.8 ns per round trip), so the thread-versus-process gap is well outside the noise. Reading it:

Quantity	pipe	futex
round trip, same process (2 switches)	1,520 ns	1,373 ns
round trip, two processes (2 switches)	1,935 ns	1,789 ns
per switch, same mm	760 ns ≈ 3,845 cycles	686 ns ≈ 3,472 cycles
per switch, different mm	967 ns ≈ 4,893 cycles	895 ns ≈ 4,525 cycles
delta attributable to the address-space switch	+207 ns ≈ +1,048 cycles	+208 ns ≈ +1,053 cycles
getppid() syscall, for scale	37 ns ≈ 187 cycles	—

The two IPC mechanisms, which share nothing but the scheduler, produce the same ~207 ns delta. That number is the price of taking the “different mm” arm of switch_mm_irqs_off() instead of its early return: the cpumask update, choose_new_asid(), the CR3 write and whatever TLB consequences follow it, plus the conditional IBPB from cond_mitigation(). It is not possible to attribute it between those components without hardware counters (see the uncertainty callout above about whether this host even has PCID), but the total is solid, reproducible, and about 27% of a same-process switch.

Also worth noticing: a full switch is ~20× the cost of a trivial syscall. The syscall entry/exit machinery is not what makes context switching expensive.

The same two peers, moved apart

Re-running the identical benchmark with the peer pinned to a different CPU changes the picture completely:

### A: same logical CPU (4,4)
  pipe  round trip, threads            1528.2 ns    7679 cyc   -> per switch  764.1 ns /  3840 cyc
  pipe  round trip, processes          1943.3 ns    9765 cyc   -> per switch  971.7 ns /  4883 cyc
### B: SMT siblings, same physical core (4,20)
  pipe  round trip, threads            3484.8 ns   17716 cyc   -> per switch 1742.4 ns /  8858 cyc
  pipe  round trip, processes          3557.0 ns   18083 cyc   -> per switch 1778.5 ns /  9041 cyc
### C: different physical cores, same L3 / CCX (4,6)
  pipe  round trip, threads            2749.4 ns   13849 cyc   -> per switch 1374.7 ns /  6925 cyc
  pipe  round trip, processes          2813.6 ns   14173 cyc   -> per switch 1406.8 ns /  7086 cyc
### D: different CCX / L3 domain (4,8)
  pipe  round trip, threads            4192.0 ns   21136 cyc   -> per switch 2096.0 ns / 10568 cyc
  pipe  round trip, processes          4107.3 ns   20709 cyc   -> per switch 2053.6 ns / 10354 cyc

(Topology confirmed from sysfs rather than assumed: cpu4’s thread_siblings_list is 4,20, so CPUs 4 and 20 are SMT siblings of one physical core; cpu4 and cpu6 share shared_cpu_list 0-7,16-23 on L3; cpu8’s L3 domain is 8-15,24-31, a different CCX. An earlier draft of this note mislabelled CPU 20 as “another CCD” — the sysfs check is what caught it.)

Placement	ns / switch (threads)	ns / switch (processes)	thread→process delta
A — same logical CPU	764	972	+208 ns
B — SMT siblings, one core	1,742	1,779	+37 ns
C — different cores, same L3	1,375	1,407	+32 ns
D — different CCX	2,096	2,054	−42 ns (noise)

Two results, and the second is the one that matters in practice.

First: once the peers are on different CPUs, the address-space switch stops mattering. The clean +208 ns signal collapses into noise — in row D it is negative. It has not gone away; it is simply dwarfed by everything else the cross-CPU wakeup path has to do: try_to_wake_up() taking a remote run-queue’s state, the smp_store_release(&prev->on_cpu, 0) handoff that lets the waker proceed, an inter-processor interrupt or a MONITOR/MWAIT idle exit on the target, and the migration of the futex word or pipe buffer cache line between cores. The lesson for anyone tuning: if a ping-pong workload is slow, task placement is worth an order of magnitude more attention than whether the peers share an address space.

Second: a switch is 2.7× costlier across a CCX boundary than on one CPU, and the cost grows monotonically with distance — except that SMT siblings (B) are slower than separate physical cores in the same L3 domain (C), which is the opposite of a naive “closer is faster” model.

Uncertain

Verify: why SMT siblings (1,742 ns/switch) are consistently slower than two different physical cores in the same L3 domain (1,375 ns/switch). Reason: the plausible explanations — the physical core dropping to a lower boost state when both SMT threads are resident, contention for shared front-end/execution resources during the wakeup, or a different idle state being entered between ping-pong halves — were not distinguished, because doing so needs hardware performance counters or turbostat, and this host has no perf installed and kernel.perf_event_paranoid is 2. To resolve: measure per-core effective frequency (APERF/MPERF) and C-state residency during each variant. uncertain

Where the time actually goes

Combining the measurements with the structure traced above, a same-CPU, same-process voluntary switch costs roughly 3,800 core cycles on this machine, and the register/stack swap itself is a vanishingly small part of it:

Component	Rough scale	Notes
__switch_to_asm: 6 pushes, 2 movs, 6 pops	tens of cycles	the part everyone pictures when they hear “context switch”
FILL_RETURN_BUFFER	tens of cycles	RSB_CLEAR_LOOPS iterations, unconditional when the feature bit is set
__switch_to: segment/FSBASE/PKRU/sp0	tens of cycles	mostly guarded by comparisons so the common case skips writes
CR3 write + TLB consequences + conditional IBPB	~1,050 cycles, measured	only on a different-mm switch; zero for thread↔thread
Scheduler proper: __schedule(), pick_next_task(), run-queue locking, finish_task_switch()	the balance — most of it	policy, not mechanics; see The EEVDF Scheduler and Per-CPU Run Queues and struct rq
Syscall entry/exit (two per switch, here)	~2 × 187 cycles	measured via getppid()
Cold caches / TLB after the switch	not measured here	the second-order cost, paid by the resumed task, not by the switch

The honest summary is that “context switch cost” is mostly not switch_to. switch_to is the irreducible core — a few dozen cycles of register and stack manipulation — wrapped in a scheduler decision, a possible address-space change, speculation mitigations, and, if the peer is elsewhere, a cross-CPU wakeup that costs more than everything else combined.

Uncertain

Verify: the per-component breakdown in the table above, beyond the two rows marked “measured”. Reason: the ~1,050-cycle address-space figure and the ~187-cycle syscall figure are direct differential measurements; the remaining rows are estimates from instruction counts and the residual, not measurements, because isolating __schedule() from switch_to() requires kernel tracing (ftrace’s sched_switch with function-graph timing) that was not run here. To resolve: trace-cmd record -p function_graph -g __schedule, or a bpftrace kfunc:__switch_to histogram, both of which need root. uncertain

Failure Modes and Subtleties

• Returning into the wrong task. Forgetting the three-argument last trick — using prev directly after the switch instead of the reassigned value — would have finish_task_switch() clean up the task this one switched away from rather than the one that actually ran before the resume, corrupting on_cpu state and reference counts. The macro exists precisely to prevent this.
• Interrupts and the rq lock. The switch runs with IRQs off and rq->lock held; the lock is not released until finish_task_switch() runs as next. A bug that takes the rq lock recursively, or enables interrupts mid-switch, deadlocks or corrupts the run queue.
• preempt_count invariant. finish_task_switch() WARNs if preempt_count() != 2*PREEMPT_DISABLE_OFFSET — the switch is entered with preemption disabled twice (once by schedule()’s wrapper, once by the rq lock); a mismatch signals a leaked or double-decremented count somewhere in the switch path.
• Lazy-TLB shootdown window. Because a kernel thread keeps a user mm loaded in CR3 while running, a CPU doing a TLB shootdown for that mm must still reach this “lazy” CPU. The accounting in mm_count (and the IPI logic) that makes this correct is detailed in The Translation Lookaside Buffer and TLB Shootdowns.
• Cost. A switch between tasks of different address spaces pays the CR3 reload (possibly mitigated by PCID) and potential TLB pressure on top of the register/stack swap; a switch within the same address space skips phase one’s expensive arm entirely via an early return. Measured on this host at +207 ns (~1,050 core cycles) per switch, reproducible across two independent IPC mechanisms — but see the next bullet before generalising it.
• Believing the thread-vs-process gap matters when the tasks are on different CPUs. Measured above: on one CPU the gap is a clean +208 ns; across CPUs it vanishes into noise, because the cross-CPU wakeup path costs 2–3× a switch all by itself. Optimising “share an address space” while ignoring placement is optimising the wrong term.
• Assuming switch_mm_irqs_off() switches the mm. Its first parameter is named unused; it reads the truth from per-CPU cpu_tlbstate, and it has two early returns that write no CR3 at all. “The scheduler called switch_mm” and “the address space changed” are different statements.
• Assuming PCID is available because the CPU is modern. choose_new_asid() degrades to ASID 0 + need_flush whenever X86_FEATURE_PCID is clear, silently turning every cross-process switch into a flushing CR3 write. Check /proc/cpuinfo for the pcid flag (not invpcid, which is a different feature) before reasoning about tagged TLBs.
• Expecting more than six address spaces to stay TLB-resident. TLB_NR_DYN_ASIDS is 6. A CPU cycling among more processes than that reallocates ASID slots round-robin and flushes anyway, so PCID’s benefit is capped by a small constant, not by the hardware’s 4,096 PCIDs.
• Attributing switch cost to switch_to. The register/stack swap is a few dozen cycles out of ~3,800. The scheduler decision, the address-space change, the speculation mitigations, and (if applicable) the cross-CPU wakeup are where the time goes.
• Timing a context switch with rdtsc and calling the result “cycles”. On this machine the TSC ticks at 2.9943 GHz while the core runs near 5.05 GHz, so that mistake under-reports by ~41%. rdtsc measures time at a fixed rate, not core cycles; the two coincide only on a machine pinned to its TSC frequency.
• Reading current inside the seam. Between the stack swap in __switch_to_asm and raw_cpu_write(pcpu_hot.current_task, next_p) in __switch_to, the CPU is running on next’s stack while current still names prev. Code injected into that window (a tracer, an instrumentation hook) that dereferences current is reading the wrong task.
• Touching prev after finish_task(). smp_store_release(&prev->on_cpu, 0) publishes prev to every other CPU’s try_to_wake_up(). After it, prev may already be running elsewhere — which is why finish_task_switch() snapshots prev->__state into a local first.

See Also

• The Core Scheduler and __schedule — the caller; __schedule() picks next and then invokes context_switch()
• The EEVDF Scheduler — which task to switch to, and why; this note owns only the mechanics of performing the switch, never the decision to make one
• TLS Models and Access — the userspace half of the FSBASE bridge: what %fs:-relative code does with the thread pointer that __switch_to restores, and the four access models that decide the offset
• The Thread Control Block — what thread.fsbase points at once it is loaded into the FS base
• Kernel Page-Table Isolation — the other reason a switch writes CR3, and why the measurements here (on a meltdown: Not affected machine) are a lower bound for PTI-enabled hosts
• The Translation Lookaside Buffer and TLB Shootdowns — CR3/PCID internals, lazy-TLB invalidation, and why skipping the flush for kernel threads is safe
• The mm_struct and Process Address Space — mm vs active_mm, the lazy-TLB mm_count reference, kernel-thread address spaces
• Migration the Stop Class and the Migration Thread — sibling; switching into the stopper that evicts a migrating task is one of these context switches
• Kernel Preemption Models — when and where a switch is allowed to happen (the preemption points that call schedule())
• Per-CPU Run Queues and struct rq — holds rq->lock (handed across the switch) and rq->prev_mm (the deferred mm drop)
• Process Exit wait and Zombie Reaping — the TASK_DEAD reaping that finish_task_switch() performs for an exiting prev
• Linux Process Scheduling MOC — parent map