vault

Cycles Per Instruction

18 min read

Cycles Per Instruction

Cycles Per Instruction (CPI) is the primary performance metric for a CPU microarchitecture: the average number of clock cycles taken to execute one instruction, averaged over some workload. It is defined for a program as CPI = total_cycles / total_instructions_retired, and it is the reciprocal of Instructions Per Cycle (IPC), the metric used by the Linux perf tool (Wikipedia, Cycles per instruction; perf-stat(1) man page). CPI plugs into the canonical CPU performance equation CPU_time = instruction_count × CPI × clock_period, which means that to make a program run faster the architect has exactly three levers: reduce the instruction count (an ISA and compiler choice), reduce the CPI (a microarchitecture choice), or reduce the clock period (a circuit and process choice). The MIT 6.004 notes call out exactly this product of three terms: “the total number of instructions executed… the average number of clock cycles it takes to execute a single instruction… and the duration of a single clock cycle” (MIT 6.004 Ch. 15). The ideal CPI of a perfect Classic Five-Stage Pipeline is 1.0; the actual CPI of a real pipeline is always 1.0 plus the average per-instruction cost of every hazard that ever fires. For the RV32IMC core built in definitely-not-esp32, CPI is the single most important number to measure, and it is measured on real hardware using the mcycle and minstret Control and Status Registers (CSRs) defined by the RISC-V Zicntr extension.

1. The CPU Performance Equation

The Wikipedia article on computer performance gives the canonical form of the equation:

T = N × C / f

where T is execution time, N is the number of instructions executed (the dynamic instruction count), C is the average CPI, and f is the clock frequency in cycles per second (Wikipedia, Computer performance). Equivalently, with P = 1/T for performance and I = 1/C for IPC,

P = I × f / N

This single equation is the start of every performance argument in computer architecture.

Walked symbol by symbol:

  • T: total wall-clock time the program takes, in seconds.
  • N: total number of dynamic instructions executed (an instruction in a loop body counts once per iteration). On a RISC-V core this is exactly the value of the minstret counter at end-of-program minus its value at start (RISC-V Unprivileged ISA, src/unpriv/zicntr.adoc).
  • C: average CPI over those N instructions, dimensionless.
  • f: clock frequency in Hz (cycles per second). For the ESP32-C3, up to 160 MHz (Espressif TRM, Ch. 1).
  • 1/f: clock period in seconds per cycle.

The product N × C is the total number of cycles the program takes, equal to mcycle’s delta. Dividing by f converts cycles to seconds.

The equation factors performance into three independent levers, each owned by a different layer of the system:

  1. Compiler and ISA choose N. A CISC instruction that does the work of three RISC instructions reduces N but typically inflates C. An aggressive optimizing compiler may eliminate redundant work, reducing N directly.
  2. Microarchitecture chooses C. Pipelining, caching, branch prediction, out-of-order execution, and superscalar issue all attack C. The job of pipeline design is essentially the job of pushing C toward 1, and (for superscalar designs) below 1.
  3. Circuits, process, and architecture together choose f. Deeper pipelines, smaller process nodes, and lower-Vt cells all push f up. But every increase in f typically degrades C (because deeper pipelines have larger mispredict penalties), so the product C × clock_period cannot be optimized independently.

This is why performance debates are never “is X fast?” but “what does X do to the product N × C / f?“

2. Mental Model: The CPI Stack

Modern microarchitects think of CPI not as a single number but as a stack: ideal CPI plus a sum of per-hazard contributions. Each kind of stall, miss, or mispredict adds its own bar to the stack.

flowchart TB
  subgraph Stack["CPI stack (illustrative, in-order 5-stage RV32 running compiled C)"]
    IDEAL["Ideal CPI: 1.00"]
    LOAD_USE["+ load-use stalls: ~0.08"]
    BRANCH["+ branch mispredict: ~0.05"]
    DCACHE["+ D-cache miss penalty: ~0.10"]
    ICACHE["+ I-cache miss penalty: ~0.03"]
    MULDIV["+ multi-cycle M-ext ops: ~0.04"]
    TOTAL["= Actual CPI: ~1.30"]
  end
  IDEAL --> LOAD_USE --> BRANCH --> DCACHE --> ICACHE --> MULDIV --> TOTAL

A representative CPI stack for a small in-order RV32 core. What it shows: each pipeline-level cost adds to the ideal CPI of 1.00; the actual CPI is the sum. The values shown are illustrative (real workloads vary wildly), but the structure is universal: every CPI report can be decomposed into “ideal” plus a list of per-hazard adders. The insight to take: optimizing the wrong term gives no improvement. If your CPI is 1.30 and 0.10 of that is D-cache misses, no amount of branch-predictor sophistication will get you below 1.20. This is the practical face of Amdahl’s law applied to CPI (Wikipedia, Amdahl’s law).

3. Mechanical Walk-through: Ideal CPI versus Actual CPI

3.1 Ideal CPI in a Pipelined CPU

In the Classic Five-Stage Pipeline, the ideal CPI is 1.0. The MIT 6.004 notes state this directly: “in the 5-stage pipelined implementation of the Beta, the hardware was designed to complete the execution of one instruction every clock cycle, so CPI_ideal is 1” (MIT 6.004 Ch. 15). The reason: at steady state, one instruction completes its WB stage every cycle. The Wikipedia article on CPI classifies CPUs by their ideal CPI (Wikipedia, Cycles per instruction):

  • Subscalar: ideal CPI > 1 (typical of unpipelined CPUs).
  • Scalar: ideal CPI = 1 (typical of in-order pipelined CPUs like the 5-stage RISC).
  • Superscalar: ideal CPI < 1 (multi-issue out-of-order CPUs like modern x86 and ARM).

For a non-pipelined multi-cycle CPU, the ideal CPI is the weighted average of per-instruction-type cycle counts. The Wikipedia article gives the formula:

CPI = Σ_i (IC_i × CC_i) / Σ_i IC_i

where IC_i is the count of instructions of type i in the workload and CC_i is the cycle count for that type. The article’s worked example sums “Integer arithmetic (45,000 instructions, 1 cycle each), Data transfers (32,000 instructions, 2 cycles each), Floating point (15,000 instructions, 2 cycles each), Control transfers (8,000 instructions, 2 cycles each)” to a weighted average of CPI = 1.55 (Wikipedia, Cycles per instruction).

This weighted-average view is also how pipeline designers think about CPI inflation: each per-instruction-class stall, even though it adds a constant cycle for that class, dilutes into the overall average proportional to the class’s frequency.

3.2 Actual CPI: The Hazard Adders

Real CPI is 1 + sum of per-instruction stall contributions. The contributions come from exactly the families covered in Pipeline Hazards:

  • Load-use hazard. Every load followed immediately by a dependent instruction costs one bubble (see Load-Use Hazard). If the workload has a load-use rate of r_lu per instruction, the contribution is r_lu × 1.
  • Branch mispredict. Every mispredict costs B bubbles, where B is the depth from IF to the branch-resolution stage. The contribution is branch_rate × mispredict_rate × B. On a 5-stage RISC-V resolving in EX, B = 2; deeper pipelines pay much more. The Wikipedia Branch predictor article gives the modern reference: “10-20 cycle misprediction penalties” in deep pipelines (Wikipedia, Branch predictor).
  • D-cache miss. A load that misses the L1 D-cache stalls until the data is filled from L2 (tens of cycles) or main memory (hundreds of cycles on a multi-GHz core; many fewer on a 160 MHz embedded core like the ESP32-C3 where memory is on-chip SRAM).
  • I-cache miss. Same idea for fetches; usually rare in code with good locality but devastating when it happens.
  • Multi-cycle functional units. The M extension’s multiply and divide, if not pipelined, hold EX for multiple cycles; floating-point ops are similar. Each adds (latency - 1) cycles per occurrence to the stall budget.

Each contribution is a product of a frequency (how often per instruction) and a cost (how many bubbles per event). The CPI stack is just the sum of these products, plus the ideal 1.0.

The Wikipedia Hazard article puts the consequence simply: “Pipeline bubbling/stalling… gives prior instructions time to complete before dependent operations begin,” which is correct but expensive in CPI (Wikipedia, Hazard (computer architecture)).

3.3 Why CPI Can Go Below 1 (Superscalar)

A superscalar CPU has multiple parallel pipelines and can complete more than one instruction per cycle in steady state. The Wikipedia Cycles per instruction article describes this as “Superscalar (CPI < 1)” (Wikipedia, Cycles per instruction). A modern dual-issue x86 core can sustain IPC of 3 or higher on integer code, which is CPI of about 0.33.

This is the regime where the metric is usually quoted as IPC rather than CPI, because IPC > 1 is more intuitive than CPI < 1. The Wikipedia article on instructions per second formalizes the conversion: IPC = 1/CPI and IPS = sockets × cores × clock × IPC (Wikipedia, Instructions per second).

For the definitely-not-esp32 project’s in-order RV32IMC core, CPI < 1 is not on the menu; the design’s CPI floor is 1.0 and the engineering goal is to keep the actual CPI close to that floor.

4. The Clock-vs-CPI Trade-off

Architectures make different choices about where to push performance: harder on clock frequency (longer pipelines) or harder on CPI (shorter pipelines, more aggressive forwarding and prediction).

The Wikipedia Computer performance article gives the textbook framing: “Speed-demon designs: prioritize improving clock frequency. Brainiac designs: focus on reducing cycles per instruction through techniques like out-of-order execution and branch prediction” (Wikipedia, Computer performance).

The clearest historical case study is Intel’s Pentium 4 NetBurst architecture. The Wikipedia article notes Prescott’s “31-stage pipeline” and clock rates reaching 3.8 GHz, ambitiously high for the era (Wikipedia, Pentium 4). The trade-off was visible in the CPI: a 31-stage pipeline pays an enormous branch-mispredict cost (potentially 31 cycles per mispredict; even small mispredict rates inflate CPI dramatically), and even with elaborate branch predictors and trace caches, the Pentium 4’s CPI on typical code was worse than its predecessor’s. The Pentium 4 was a clock-frequency-first design that found the wall: more pipeline depth eventually destroys CPI faster than it buys frequency, and Intel abandoned NetBurst for the shorter-pipeline Core microarchitecture (Wikipedia, Pentium 4).

The opposite end is the ARM Cortex-M0+ with its 2-stage pipeline: every cycle of frequency it gives up (compared to a 5-stage core) it more or less gets back in IPC, because the load-use bubble disappears and the branch-mispredict cost is zero or one cycle. For battery-powered MCUs, the Cortex-M0+‘s low CPI at modest clocks is a far better fit than a high-clocked design with worse CPI.

The ESP32-C3, with its 4-stage in-order RISC-V pipeline at 160 MHz, sits in the middle: optimized “for area, power and performance” (Espressif TRM, Ch. 1). The choice of 4 stages (versus the textbook 5) buys slightly better CPI at a slightly lower achievable clock; the trade-off is reasonable for a Wi-Fi-BLE MCU where the radio peripherals, not the core, dominate energy.

5. Measuring CPI on Real Hardware

5.1 The RISC-V Mechanism: mcycle and minstret

The RISC-V ISA defines two hardware performance counters that exactly capture the two terms needed to compute CPI:

  • cycle (and its machine-mode shadow mcycle): “counts … the number of clock cycles executed by the processor core” (RISC-V Unprivileged ISA, src/unpriv/zicntr.adoc). Accessed by user code via the rdcycle pseudoinstruction.
  • instret (machine-mode minstret): “counts the number of instructions retired by this hart from some arbitrary start” (RISC-V Unprivileged ISA, src/unpriv/zicntr.adoc). Accessed by rdinstret. Importantly, “instructions causing synchronous exceptions don’t retire,” so the count excludes traps that did not commit.
  • Both counters are 64 bits wide, “to prevent undetectable overflow, even on XLEN=32 implementations.” On a 32-bit RV32 core like the definitely-not-esp32 target, the upper 32 bits are read separately via rdcycleh and rdinstreth.

These are the Zicntr extension. They are mandatory in any RISC-V profile that targets serious software (the user-mode portion of the standard application profiles). The definitely-not-esp32 M-mode-only microkernel can use the machine-mode aliases mcycle and minstret directly.

The measurement pattern is:

start_cycle  = rdcycle()
start_instret = rdinstret()
... workload ...
end_cycle    = rdcycle()
end_instret  = rdinstret()
cpi = (end_cycle - start_cycle) / (end_instret - start_instret)

For longer measurements that might overflow the 32-bit lower halves on RV32, the standard trick is to read low, read high, re-read low, and check if low overflowed during the high read; if so, retry. This is the same race-free pattern used for any pair of 32-bit counter halves.

For more detailed CPI breakdown (cache misses, branch mispredicts, etc.), the Zihpm extension provides “up to 29 additional unprivileged 64-bit hardware performance counters” (hpmcounter3 through hpmcounter31) plus event-selector CSRs that map each counter to an implementation-defined microarchitectural event (RISC-V Unprivileged ISA, src/unpriv/zihpm.adoc). The Zihpm spec is explicit that “the number, width, and functionality of these counters are platform-specific” and that “the configuration used to select the events counted… is misconfigured, the counter may return a constant value.” Real CPUs (the ESP32-C3 included) implement a small subset; the kernel learns which by probing.

5.2 The Linux Mechanism: perf stat

On a Linux box (or, for the definitely-not-esp32 project, on a host running a benchmark), the standard tool is perf stat. The Wikipedia article on perf describes it as supporting “hardware performance counters, tracepoints, software performance counters” (Wikipedia, Perf (Linux)). The perf-stat(1) man page documents the default event set, which includes “cycles:u” and “instructions:u,” and notes that perf “calculates and displays instructions per cycle in the right-most column” (perf-stat(1)).

A canonical invocation:

$ perf stat -e cycles,instructions ./my_program
...
   229,570,665,834   cycles:u            #    2.742 GHz
   313,163,853,778   instructions:u      #    1.36  insn per cycle

The 1.36 insn per cycle is IPC; the CPI is 1 / 1.36 = 0.735. Brendan Gregg’s perf reference confirms the interpretation: “Higher IPC values mean higher instruction throughput, and lower values indicate more stall cycles” and “high IPC values (e.g., over 1.0) as good, indicating optimal processing of work” (Brendan Gregg, Linux perf Examples).

The Wikipedia article on hardware performance counters notes the general implementation constraint: counters are “special-purpose registers built into modern microprocessors to store the counts of hardware-related activities,” and “the limited number of registers often requires conducting multiple measurements to collect all desired metrics” (Wikipedia, Hardware performance counter). Perf handles this for the user via multiplexing: if more events are requested than the hardware can count simultaneously, perf time-slices the counter assignments and scales the reported counts.

For richer breakdowns, perf stat -d adds L1 and last-level cache events; perf stat -e cache-misses,branch-misses,instructions,cycles lets you compute the per-instruction stall contributions and reconstruct the CPI stack from the bottom up.

6. Worked Example: CPI Math on a Small RV32 Loop

Consider this loop (one of the test workloads for the definitely-not-esp32 core):

loop:
  lw   x5, 0(x10)         # 1 instr; load array[i]
  add  x6, x6, x5         # 1 instr; load-use, 1 bubble
  addi x10, x10, 4        # 1 instr
  bne  x10, x11, loop     # 1 instr; branch, 2 bubbles if mispredicted

Per iteration: 4 instructions plus stall budget. Suppose the data is hot in the L1 D-cache (one-cycle hit) and the branch is taken 95% of the time, but the branch predictor (a Two-Bit Saturating Counter) predicts “taken” correctly 98% of the time. Then per iteration:

  • Load-use bubble: 1 cycle (always, since the compiler did not reorder).
  • Branch mispredict: 0.02 × 2 = 0.04 cycles on average.
  • Total per iteration: 4 + 1 + 0.04 = 5.04 cycles for 4 instructions.
  • CPI: 5.04 / 4 = 1.26.

This matches the CPI-stack model: ideal 1.0, load-use adder 0.25 (one bubble in four instructions), branch-mispredict adder 0.01. The total of 1.26 is reasonable for a small in-order RV32 core; aggressive scheduling by the compiler could bring it back near 1.0, but only at the cost of code-size growth (the scheduled version needs more instructions to maintain liveness).

The MIT 6.004 notes summarize the equation in terms specific to a pipelined design: “bypassing and branch prediction ensure that data and control hazards have only a small negative impact on the effective CPI” (MIT 6.004 Ch. 15). The CPI-stack arithmetic is the quantitative version of that statement.

7. CPI as a Comparison Metric: Limits and Pitfalls

CPI is a microarchitecture metric, not a performance metric. Two CPUs with the same CPI can run the same program at radically different speeds because their clock frequencies differ. Two CPUs with very different CPIs can run the same program at the same speed because the higher-CPI one has a much higher clock. Always quote CPI alongside clock frequency.

CPI is also workload-dependent. A CPI of 1.0 on a tight integer kernel says nothing about CPI on a memory-bound benchmark or on a floating-point matrix multiplication. The Wikipedia article on instructions per second cautions: “instructions/cycle measurement depends on the instruction sequence, the data and external factors” (Wikipedia, Instructions per second). The honest way to report CPI is to publish the CPI on a named benchmark with a named compiler at a named optimization level.

CPI across ISAs is meaningless without instruction-count normalization. A CISC instruction that does the work of three RISC instructions inflates the RISC machine’s N, depresses its CPI (because each RISC instruction is simpler, less likely to stall), and otherwise scrambles the comparison. The product N × C / f is the only thing that survives ISA-to-ISA comparison; that product, in seconds, is execution time, the only number that ultimately matters.

8. Failure Modes and Common Misunderstandings

  • “CPI 1.0 means the pipeline is ideal.” It means the pipeline is at its ideal CPI floor, which for a single-issue in-order pipeline is 1.0. It says nothing about whether you are using that pipeline well; an idle CPU has CPI of 1.0 on its NOPs too.
  • “Lower CPI is always better.” Not if it costs frequency. A halving of CPI bought by a tripling of clock period is a net loss. The CPU performance equation is a product.
  • “Pipelining reduces CPI.” Pipelining reduces ideal CPI from > 1 (for a multi-cycle unpipelined CPU) to 1. It does not reduce CPI below 1; that takes superscalar issue.
  • “perf’s IPC is exactly the same as architectural CPI.” Close but not identical. The instructions counter is “instructions retired,” which can differ from “instructions dispatched” or “instructions issued” by orders of magnitude on a speculative out-of-order core. For an in-order RV32 like the definitely-not-esp32 target, instructions retired equals instructions executed equals minstret, and the difference vanishes.
  • “CPI predicts power.” It does not, directly. Two designs with the same CPI can burn very different power; pipeline depth, voltage, and activity factor all matter. The relationship between performance and power is the subject of separate metrics like Energy-Delay Product, not CPI alone.
  • “Hardware counters are perfectly accurate.” They are not. The Wikipedia article warns of the “limited number of registers” and the need for multiplexing, and the perf documentation notes that some derived metrics are statistical samples rather than exact counts (Wikipedia, Hardware performance counter; Brendan Gregg, perf). For the basic mcycle and minstret counters in RISC-V, the count is exact (modulo the documented “instructions causing synchronous exceptions don’t retire” rule).
  • “You can derive CPI from MIPS rating.” Only if you also know the clock and the instruction-mix workload. MIPS = f / CPI / 1e6, so to recover CPI you need both f and MIPS. The “MIPS rating” of a CPU is itself ambiguous unless the workload that produced it is named (DMIPS, native MIPS, MIPS at peak burst, etc.).

9. Production Notes for the definitely-not-esp32 Project

For the project’s benchmarking phase, the CPI measurement protocol is:

  1. Implement Zicntr in the core. Add mcycle/minstret CSRs to the CSR file. mcycle increments every cycle the core is not gated off; minstret increments at the end of every WB stage in which a non-bubbled, non-trapped instruction commits.
  2. Expose rdcycle/rdinstret to user code. The pseudoinstructions are just csrrs rd, cycle, x0 (read-and-OR-with-zero). For M-mode-only kernels, the user-mode aliases can be omitted; M-mode reads mcycle and minstret directly.
  3. Define the benchmark suite. A small set of named kernels: integer matrix multiply, AES block encryption, a linked-list traversal (load-latency-bound), a recursive Fibonacci (branch-heavy), an integer fast-Fourier transform. Each kernel becomes one CPI data point.
  4. Compare side-by-side with the ESP32-C3. Compile the same Rust benchmark to both targets, run on both, and report the table of cycles, instructions, CPI, and wall-clock time. The CPI gap tells you exactly how much the definitely-not-esp32 core’s choices (5 stages vs the ESP32-C3’s 4, hand-written forwarding vs SiFive E2’s) cost or save per instruction.
  5. Plot the CPI stack. Use the available hpmcounter events to attribute the CPI gap to specific stall classes (load-use bubbles, branch mispredicts, multi-cycle multiplies). The honest CPI stack is the deliverable; a single CPI number without attribution is not very useful.

10. See Also

Graph View

Backlinks