Pipeline Hazards

A pipeline hazard is a situation in which the pipeline cannot, in the very next cycle, fire the instruction that should logically fire next, because doing so would either produce a wrong answer or require a resource that is not free. The CMU 18-447 lecture puts the framing in two sentences that should be memorized: “dependence is a property of the program; hazards are specific to the microarchitecture” (CMU 18-447 Lecture 8). Hazards come in exactly three families: structural (two instructions want the same hardware in the same cycle), data (an instruction needs an operand a still-in-flight instruction is producing), and control (the address of the next instruction depends on a branch that has not resolved). A correct pipelined CPU must detect and resolve every one of them, either by inserting bubbles (stalls), forwarding values, predicting branches, or, in more aggressive designs, renaming registers and reordering instructions. In an in-order Classic Five-Stage Pipeline like the one used in the definitely-not-esp32 project, only one of the three data-hazard subtypes (Read After Write, or RAW) is actually a problem; the other two are eliminated by the in-order design.

1. Mental Model: Why Hazards Exist

The simplest way to see why hazards arise is to look at the steady-state space-time diagram of a 5-stage pipeline. Five instructions are in flight, each in a different stage. The fact that they all coexist means that the result of an instruction in EX, MEM, or WB has not yet been committed to the architectural register file when the next instruction enters ID and reads from that file. If the next instruction’s source register is the previous instruction’s destination, naive register-file lookup returns the stale value. That is a data hazard, specifically a RAW hazard. If the next instruction is a branch and we have not yet resolved its outcome, we do not even know the address of the next next instruction. That is a control hazard. If two stages need the same memory port, we cannot run both. That is a structural hazard.

flowchart LR
  subgraph Program["Programmer's view (no hazards)"]
    P1["i1: x5 = ..."] --> P2["i2: x6 = x5 + 1"]
  end
  subgraph Pipeline["Pipeline view (hazard window)"]
    direction LR
    A1["i1 in EX/MEM/WB:<br/>writing x5"]
    A2["i2 in ID:<br/>reading x5"]
    A1 -. "i2 reads x5 before i1 writes it" .-> A2
  end

The dependence-vs-hazard gap. What it shows: the programmer sees one logical update of x5 followed by a read of x5 (no problem). The pipeline sees them overlapped in time, so unless something extra is done (forwarding or stalling), the read in ID sees an old value. The insight to take: the program is correct; the implementation introduces the hazard. That is the meaning of “hazard is a property of the microarchitecture.”

2. The Three Hazard Families, In Order

2.1 Structural Hazards

A structural hazard occurs “when two instructions might attempt to use the same resources at the same time” (Wikipedia, Classic RISC pipeline). In the canonical 5-stage pipeline the textbook structural hazard is single-port memory: if IF and MEM both have to read or write the same memory in the same cycle, one of them must wait. The textbook solution is to declare the pipeline Harvard at the L1 boundary, separating the I-port from the D-port and avoiding the conflict by construction (Wikipedia, Classic RISC pipeline). When that is not possible (a small embedded core sharing a single SRAM, for example), an explicit stall has to gate IF off whenever MEM is using the bus.

A second structural hazard, often glossed over, is register-file port contention: WB writes the register file in the same cycle that ID reads it. The fix described by Wikipedia is to clock the file so that “writes happen on the falling edge and reads on the rising edge,” so the read sees the write from earlier in the same cycle (Wikipedia, Classic RISC pipeline). The CMU 18-447 lecture lists the three generic structural-hazard fixes: “duplicating the resource, increasing its throughput, or detecting the resource contention and stalling one of the stages” (CMU 18-447 Lecture 8). The 5-stage pipeline canonically applies duplicate-the-resource (separate I- and D-memory) and increase-throughput (dual-edge register file) so that no stalls are needed.

Multi-cycle functional units (an unpipelined integer multiplier in the M extension, a divider, a floating-point unit) also create structural hazards if a new instruction wants the unit before the previous one has freed it. The textbook resolution is to stall the EX stage until the unit is free, which is logically a structural-hazard stall.

2.2 Data Hazards: The Three Subtypes

Data hazards come from data dependences between instructions. The CMU lecture, citing the program/microarchitecture distinction, distinguishes three dependence types (CMU 18-447 Lecture 8):

• Read After Write (RAW), also called true dependence or flow dependence. Instruction i writes a register; instruction j (later in program order) reads it.

  i: x3 <- x1 op x2
  j: x5 <- x3 op x4   (j needs the value i produced)

  RAW is a true ordering requirement: any correct execution must read x3 after i has written it.

• Write After Read (WAR), also called anti-dependence. Instruction i reads a register; instruction j writes it.

  i: x3 <- x1 op x2   (i reads x1)
  j: x1 <- x4 op x5   (j writes x1)

  WAR is a name dependence: the two instructions are reusing the same register name, but the data they actually need is unrelated. With a different register name, there would be no dependence.

• Write After Write (WAW), also called output dependence. Both instructions write the same register.

  i: x3 <- x1 op x2
  j: x3 <- x6 op x7

  Also a name dependence; the program semantics demand that the final architectural value of x3 come from j, not i, but otherwise the writes are independent.

These three categories are taken directly from the data-dependence vocabulary used by every compiler and architecture textbook (Wikipedia, Data dependency; CMU 18-447 Lecture 8).

Why an in-order pipeline only sees RAW as a real problem

In an in-order, single-issue pipeline like the Classic Five-Stage Pipeline, WAR and WAW dependences cannot actually create a hazard. Reasoning: instructions enter ID in program order and write back in program order. The reader (in WAR) is always older than the writer, so it reaches ID first and reads its operand before the writer’s WB happens; there is no way for the write to overtake the read. Likewise the two writes (in WAW) reach WB in program order; the later write always commits last, which is what the semantics demand. The Wikipedia hazards article puts the same point in the negative: WAR and WAW are problems only when “the actual execution order in the pipeline is not the program order,” which is exactly the situation in out-of-order processors and is exactly what techniques like scoreboarding and Tomasulo’s algorithm exist to fix (Wikipedia, Tomasulo algorithm). For the in-order RV32IMC core built in definitely-not-esp32, we only have to think about RAW data hazards.

When does a RAW dependence become a RAW hazard?

It depends on the pipeline’s geometry. In the 5-stage pipeline, a register is written in WB (cycle t+4 for an instruction whose IF was in cycle t) and read in ID (cycle t+1 for an instruction whose IF was in cycle t). So a RAW dependence is a hazard whenever the distance in instructions between the writer and the reader is less than four: distances 1, 2, and 3 create hazards; distance 4 or more does not, because by then the writer has reached WB.

The CMU lecture summarizes this as dist_dependence(i, j) <= dist_hazard(X, Y) ⇒ hazard, where X is the writing stage and Y is the reading stage (CMU 18-447 Lecture 8). For RISC-V on the 5-stage pipeline, the writer’s stage is WB and the reader’s stage is ID, so the hazard distance is 3. Without forwarding, this means three bubbles are needed between a producer and any of the next three consumers. With forwarding, this collapses to zero bubbles for non-load producers, and to one bubble for a load producer (the load-use hazard).

2.3 Control Hazards

A control hazard arises whenever the next-PC of the pipeline depends on the outcome of an instruction that has not yet completed. The Wikipedia hazards article frames it succinctly: “control hazard occurs when the control logic incorrectly predicts which program branch will be taken” and the pipeline ends up holding instructions that should never have been fetched (Wikipedia, Hazard). In the 5-stage pipeline the question is: which stage resolves a branch?

• If the branch resolves in EX (one common design), then IF and ID have already fetched and decoded two instructions on the predicted path. A misprediction costs two bubbles.
• If the branch resolves in MEM (the CS61C 5-stage RISC-V variant resolves in MEM), then three instructions have been speculatively fetched on the wrong path, and a misprediction costs three bubbles (CS61C Notes, Five-Stage Pipeline).
• If the branch resolves in ID (more aggressive, requires a dedicated comparator and target adder in ID), the misprediction cost is one bubble. Wikipedia notes that the Stanford MIPS approach was to resolve in ID by adding “a dedicated branch target adder in the decode stage rather than sharing the ALU” (Wikipedia, Classic RISC pipeline).

The misprediction penalty becomes the dominant performance factor as pipelines deepen. The Wikipedia branch-predictor article notes that “modern pipelines have 10-20 cycle misprediction penalties” in deeper out-of-order designs (Wikipedia, Branch predictor). For Intel’s Pentium 4 with its 31-stage NetBurst pipeline, the penalty was roughly 31 cycles, which is why every clock cycle of branch-predictor accuracy mattered enormously (Wikipedia, Pentium 4).

3. The Mitigation Menu

There are exactly six tools in the architect’s box for resolving hazards. They are not all in any one design; an in-order educational RV32 core uses two or three of them, an aggressive out-of-order x86 core uses all six.

3.1 Pipeline Stalling (Bubble Insertion)

The universal hazard resolution: when in doubt, stall. The CMU lecture calls this “universal hazard resolution” and reduces it to two mechanical steps: “stop all up-stream stages; drain all down-stream stages” (CMU 18-447 Lecture 8). The pipeline mechanically refuses to advance PC and IF/ID for one cycle, and the IF/ID-to-ID stage register is replaced with a NOP, called a bubble. The bubble then propagates down the pipeline like any other instruction, but it has no effect when it reaches EX, MEM, or WB.

Wikipedia’s pipeline-stall article describes the result with the familiar image: the bubble is “compared to an air pocket traveling through a fluid pipe” (Wikipedia, Pipeline stall). Stalls are correct in every case but expensive: each bubble is one wasted cycle. They are the resolution of last resort.

3.2 Operand Forwarding (Bypassing)

For RAW data hazards, the pipeline can usually deliver the value the consumer needs without stalling, by routing the producer’s result from the pipeline register it currently sits in directly to the consumer’s ALU input. The Wikipedia operand-forwarding article describes the mechanism: “Instead of waiting for results to be written to registers, the output of one instruction can be used by subsequent instructions before the value is committed to / stored in the register file” (Wikipedia, Operand forwarding). The CS61C course notes list the two canonical paths in a 5-stage RISC-V pipeline as “MEM to EX and WB to EX,” implemented as “wires from the output of the ALU and from the MEM/WB pipelined register that connect to the A/B muxes in the EX stage” (CS61C Notes, Five-Stage Pipeline).

Forwarding eliminates the stall cost of almost all RAW hazards. The one residual hazard it cannot fix is the load-use hazard, because a load’s data is not available until the end of the MEM stage, and a dependent ALU op needs it at the start of EX in the very next cycle, which would require time-travel. The full mechanism is the subject of Operand Forwarding; the residual case is the subject of Load-Use Hazard.

3.3 Branch Prediction

For control hazards, the textbook countermeasure is to predict the branch direction (and target) before the branch resolves, fetch speculatively along the predicted path, and flush the speculative work if the prediction was wrong. The simplest static predictor is “predict not taken,” which is correct for any fall-through path; static “predict-backward-taken-forward-not-taken” is a common improvement that captures loop semantics. Dynamic prediction uses runtime branch history; the canonical small dynamic predictor is the Two-Bit Saturating Counter, which “improves accuracy because a conditional jump has to deviate twice from what it has done most in the past before the prediction changes” (Wikipedia, Branch predictor). The full story is in Branch Prediction.

3.4 Branch Delay Slots (ISA-level Mitigation)

A historical alternative to prediction was the branch delay slot: the architecture defines the instruction after a branch to execute regardless of whether the branch is taken, so the slot after a branch is always usable cycle. The MIPS R2000 had a single branch delay slot (Wikipedia, MIPS architecture). The MIT 6.004 lecture explains why this approach has been abandoned: “only half the time can we find instructions to move to the branch delay slot … it turns out that branch prediction works better than delay slots” (MIT 6.004 Ch. 15). The Wikipedia article on delay slots gives the modern verdict by enumeration: “ARM, PowerPC, RISC-V” have no delay slots; only legacy MIPS, SPARC, SuperH, and a few DSPs retain them (Wikipedia, Delay slot). RISC-V’s choice to omit the delay slot is deliberate; it preserves implementation freedom (a deeper pipeline does not need a different ISA), and it avoids the compiler’s awkward NOP-filling problem.

3.5 Scoreboarding

When out-of-order issue is on the table, more sophisticated bookkeeping is needed. Scoreboarding is the CDC 6600’s centralized approach: a table tracks which functional units are busy and which architectural registers are pending writes, and the scoreboard releases each instruction to read operands, execute, and write back only when its hazards are clear. The Wikipedia article frames the key invariant: “reads proceed in the absence of write hazards, and writes proceed in the absence of read hazards” (Wikipedia, Scoreboarding). Scoreboarding handles WAW and WAR by stalling (not renaming), which limits its parallelism.

3.6 Register Renaming (Tomasulo’s Algorithm)

The most powerful technique: eliminate WAR and WAW entirely by giving each writer of an architectural register a fresh physical register, then renaming every subsequent reader to use the right physical register. Tomasulo’s algorithm at IBM in 1967 was the first implementation, in the System/360 Model 91 floating-point unit (Wikipedia, Tomasulo algorithm). The Wikipedia article on Tomasulo notes that “WAW & WAR hazards: Eliminated via hardware register renaming during the issue stage.” The technique underlies essentially every modern high-performance CPU, including Intel and AMD x86-64 chips and high-end ARM and RISC-V cores. It is fundamentally an out-of-order-execution technology, far beyond the scope of a 5-stage in-order pipeline.

4. Worked Examples in RISC-V Assembly

These examples target the 5-stage in-order pipeline used in definitely-not-esp32. Time runs left to right in cycles; stages are abbreviated.

4.1 Hazard-free baseline

addi x5, x0, 1     # x5 = 1
addi x6, x0, 2     # x6 = 2
addi x7, x0, 3     # x7 = 3
 
Cycle:    1    2    3    4    5    6    7
addi x5: IF   ID   EX   MEM  WB
addi x6:      IF   ID   EX   MEM  WB
addi x7:           IF   ID   EX   MEM  WB

No instruction reads a register the previous one writes; the pipeline runs at one instruction per cycle. CPI = 1.

4.2 RAW hazard resolved by forwarding

addi x5, x0, 10
add  x6, x5, x5    # x6 = x5 + x5 = 20
 
Cycle:    1    2    3    4    5    6
addi x5: IF   ID   EX   MEM  WB
add  x6:      IF   ID   EX   MEM  WB

At cycle 4, add x6 is in EX needing the value of x5. The producing addi x5 is in MEM, with its ALU result 10 already in the EX/MEM pipeline register from end of cycle 3. The EX/MEM-to-EX forwarding mux feeds 10 straight to the ALU’s input. No bubble. CPI = 1.

4.3 Load-use hazard, forwarding cannot help

lw  x5, 0(x10)     # load from memory pointed to by x10
add x6, x5, x7     # immediately use the loaded value
 
Cycle:    1    2    3    4    5    6    7
lw  x5:  IF   ID   EX   MEM  WB
add x6:       IF   ID   --   EX   MEM  WB    (-- = bubble)

The load’s data is available at the end of cycle 4 (MEM). The dependent add is in ID in cycle 3 and would otherwise enter EX in cycle 4, before the load’s data exists. A one-cycle stall is inserted: add stays in ID for an extra cycle (cycle 4), with a bubble pushed into EX. In cycle 5, add enters EX, the value of x5 is forwarded from MEM/WB, and execution proceeds. The full mechanism is in Load-Use Hazard. CPI for this two-instruction sequence is 1.5 (3 cycles for 2 instructions, in steady-state pipeline accounting).

4.4 Control hazard from a taken branch

beq  x5, x0, target    # branch if x5 == 0
addi x6, x6, 1         # speculatively fetched (fall-through)
addi x7, x7, 1         # speculatively fetched (fall-through)
...
target:
sub  x8, x9, x10
 
Cycle:        1    2    3    4    5    6    7
beq:         IF   ID   EX   MEM  WB
addi x6:          IF   ID   (squash)
addi x7:               IF   (squash)
sub x8:                     IF   ID   EX   MEM

Assuming branch resolution in EX (cycle 3), two fall-through instructions (addi x6 in IF at cycle 2, addi x7 in IF at cycle 3) have been speculatively fetched. When the branch outcome becomes known at the end of cycle 3 and turns out to be “taken,” both speculative instructions are annulled (turned into NOPs in the pipeline registers), and IF in cycle 4 is redirected to fetch the branch target sub x8. The penalty is two cycles per taken branch (cycles 2 and 3 of useful fetch wasted). With a predictor that guesses “taken with the right target,” the cost can be reduced to zero when the prediction is correct.

4.5 Structural hazard from shared memory

If the design used a single memory port (no Harvard split), any cycle in which a load or store is in the MEM stage forces a structural conflict with whatever instruction is in IF at the same cycle. Consider:

i1: sw  x5, 0(x10)
i2: addi x6, x6, 1
i3: addi x7, x7, 1
i4: addi x8, x8, 1     # this is the IF that collides with i1's MEM
 
Cycle:    1    2    3    4    5    6
i1 sw:   IF   ID   EX   MEM  WB
i2:           IF   ID   EX   MEM  WB
i3:                IF   ID   EX   MEM
i4:                     IF   ID   EX     # IF in cycle 4 collides with i1's MEM in cycle 4

i1’s MEM stage and i4’s IF stage both demand the memory port in cycle 4. One must wait; the typical choice is to stall IF (and freeze PC) for a cycle so the memory transaction completes, then resume fetching. More generally, every cycle in which a load or store sits in MEM is a cycle in which the simultaneous IF cannot proceed on a shared-port design. Modern designs avoid this by splitting I-memory and D-memory into separate ports (separate L1 caches; or, on the Tang Nano 20K FPGA target, separate block RAMs).

5. Hazard Detection: The Hardware

The hazard-detection logic in a 5-stage in-order pipeline is small and entirely combinational. It lives between the ID and EX stages and works by comparing fields of the in-flight instructions. The CS61C course summarizes it as adding “wires from the output of the ALU and from the MEM/WB pipelined register that connect to the A/B muxes in the EX stage,” driven by a small comparator network (CS61C Notes, Five-Stage Pipeline).

The data-hazard detection compares:

• The rs1 and rs2 fields of the instruction in ID against the rd field of the instruction in EX/MEM (one-cycle-old producer) and MEM/WB (two-cycle-old producer).
• If they match and the producer’s RegWrite control bit is set, drive the EX-stage operand mux to take the forwarded value.

The load-use detection is a separate small piece of logic that fires when the producer in EX/MEM is a load and the consumer in ID needs that register. It asserts a Stall signal that freezes PC and IF/ID and inserts a NOP into ID/EX. (Full diagram in Load-Use Hazard.)

The control-hazard handling is in the IF stage: a misprediction signal from the branch-resolution stage causes IF (and any stages between IF and the resolving stage) to invalidate the speculative fetches and redirect PC to the correct target.

The CMU lecture makes the point that this logic, while small, is highly specific to the pipeline’s exact depth and stage assignment (CMU 18-447 Lecture 8). Changing the pipeline from “branch resolves in EX” to “branch resolves in MEM” changes the misprediction penalty and the comparator network. This is why a pipeline-level redesign is a real piece of work, not a parameter change.

6. Why RISC ISAs Make Hazards Easier

The classic RISC ISAs (MIPS, ARM, RISC-V) were designed with pipelining in mind, which is why the 5-stage pipeline maps so cleanly onto them. Three specific RISC features make hazard handling tractable:

1. Fixed-length instructions. IF can fetch one instruction per cycle without variable-length decoding, eliminating a class of structural hazard on the fetch path. The Wikipedia RISC article identifies this as a design goal: “instructions to be fixed-length, simplifying pipelines” (Wikipedia, RISC).
2. Load-store architecture. Only loads and stores access memory. Arithmetic instructions never have a hidden memory access, so EX always takes exactly one cycle for arithmetic. This is what keeps the EX/MEM split clean.
3. Three-operand register-register instructions. RAW dependences are explicit in the encoding; the hazard detector just looks at fields rs1, rs2, rd. Variable-operand CISC instructions hide their register usage in addressing modes, complicating the detector.

The RV32IMC subset used in definitely-not-esp32 has all three properties. The base RV32I spec is explicit: “All are a fixed 32 bits in length,” instruction-aligned to four-byte boundaries, and “only load and store instructions access memory and arithmetic instructions only operate on CPU registers” (RISC-V Unprivileged ISA, src/unpriv/rv32.adoc). The compressed (C) extension adds 16-bit encodings, but its stated purpose is to “substantially improve code density” (RISC-V Unprivileged ISA, src/unpriv/c-st-ext.adoc) and each 16-bit form maps one-to-one to a 32-bit RV32I instruction at the decoder, so the hazard detector still sees a uniform stream of 32-bit operations.

7. CPI Impact

Every bubble inserted by hazard resolution costs one cycle of CPI inflation. In the ideal pipeline, CPI = 1. With one bubble per ten instructions (a not-unreasonable rate for a small in-order RV32 core running compiled C code with average load fraction and branch fraction), actual CPI is 1.1. The arithmetic is simple: actual_CPI = ideal_CPI + sum_over_hazard_types(stall_cycles_per_event * events_per_instruction). The structural breakdown is the heart of pipeline performance analysis, covered in detail in Cycles Per Instruction.

8. Failure Modes and Common Misunderstandings

• “Forwarding solves all data hazards.” No: the load-use hazard always needs one bubble; multi-cycle ALU ops can need more. (Wikipedia, Classic RISC pipeline)
• “WAR and WAW are universal hazards.” Only in out-of-order pipelines. An in-order, single-issue pipeline experiences only RAW; in-order issue trivially preserves both WAR and WAW ordering (CMU 18-447 Lecture 8).
• “Predicting branches eliminates control hazards.” It eliminates the cost of correctly-predicted branches. Mispredictions still cost the full pipeline-flush penalty, which in deep pipelines is the dominant performance ceiling (Wikipedia, Branch predictor).
• “Delay slots are a clever solution.” They were a forced solution to a specific 1980s implementation pain (no time for hardware-based hazard handling). Modern ISAs unanimously chose to omit them (Wikipedia, Delay slot).
• “A correctly designed pipeline never stalls.” Even with full forwarding and perfect prediction, cache misses, multi-cycle ops, and load-use hazards will stall the pipeline; the goal is to keep stall rate small, not zero.

9. See Also

• Classic Five-Stage Pipeline: the canonical pipeline this note’s hazards arise in.
• Operand Forwarding: the standard RAW resolution.
• Load-Use Hazard: the residual RAW hazard forwarding cannot fix.
• Cycles Per Instruction: how hazards inflate CPI.
• Branch Prediction and Two-Bit Saturating Counter: control-hazard resolution.
• RV32IMC and RISC-V Instruction Set Architecture: the ISA’s role in making hazards tractable.
• ESP32-C3: a 4-stage in-order pipeline with the same hazard taxonomy.
• Computer Architecture MOC: parent map.