Pipeline Hazards

Introduction

If pipelining were perfect, every pipeline stage would be busy every cycle, and throughput would equal one instruction per cycle. But reality is messier. Pipeline hazards are situations where the next instruction cannot execute in its scheduled cycle — the pipeline must stall, creating "bubbles" (wasted cycles). Understanding hazards and their solutions is central to pipeline design.

What is a Hazard?

A hazard is a condition that prevents an instruction from executing in its designated pipeline stage. When a hazard is detected, the pipeline must:

• Stall (insert bubble): Halt one or more stages for a cycle
• Forward/Bypass: Route data through shortcuts (for data hazards)
• Flush: Discard partially-executed instructions (for control hazards)

Structural Hazards

What They Are

A structural hazard occurs when two instructions need the same hardware resource in the same cycle.

Classic Example: Single Memory

If instruction fetch (IF) and data access (MEM) both need memory in the same cycle:

Solutions

1. Separate instruction and data memory (Harvard-style caches) — most common
2. Duplicate the resource (multiple ALUs, multiple ports)
3. Stall the pipeline — simple but reduces performance

Modern processors eliminate most structural hazards through resource duplication.

Data Hazards

What They Are

A data hazard occurs when an instruction depends on the result of a previous instruction that hasn't completed yet.

Types of Data Dependencies

RAW (Read After Write) — the most common and problematic:

WAR (Write After Read): A later instruction writes a register before an earlier one reads it. Rare in simple pipelines but relevant in out-of-order execution.

WAW (Write After Write): Two instructions write the same register — the second must write last. Also relevant in out-of-order execution.

Data Hazard Solutions

1. Stalling (Pipeline Interlock) Insert NOPs (bubbles) until the data is ready:

Simple but wastes 2 cycles.

2. Forwarding (Bypassing) — preferred solution Route the result directly from where it's produced to where it's needed, without waiting for write-back:

Zero stall cycles for most ALU-to-ALU dependencies!

3. Compiler Scheduling (Software) Reorder instructions to avoid hazards:

Control Hazards (Branch Hazards)

What They Are

A control hazard occurs when the pipeline has fetched instructions after a branch, but the branch outcome isn't known yet — the wrong instructions may be in the pipeline.

The Problem

If the branch IS taken, the two fetched instructions must be discarded (flushed), wasting 2 cycles.

Branch Penalty

The number of wasted cycles on a mispredicted branch = number of pipeline stages between fetch and branch resolution.

• 5-stage pipeline: 1-2 cycle penalty
• 20-stage pipeline: 15+ cycle penalty (much worse!)

Solutions

1. Branch Prediction Guess whether the branch will be taken:

• Static: Always predict not-taken, or predict backward branches taken (loops)
• Dynamic: Use history of past branch behavior to predict future branches
• Modern CPUs achieve 95-97% prediction accuracy!

2. Branch Delay Slot (MIPS) The instruction AFTER the branch always executes (regardless of branch outcome):

Compiler fills the delay slot with a useful instruction.

3. Speculative Execution Predict the branch direction and continue executing speculatively. If prediction wrong, flush the pipeline and restart from the correct path.

Hazard Summary

Performance Impact

Example: If 20% of instructions are branches and 10% of those are mispredicted with 3-cycle penalty:

Key Takeaways

1. Pipeline hazards prevent achieving ideal 1 instruction/cycle throughput
2. Structural hazards: Fixed by duplicating resources (separate I-cache and D-cache)
3. Data hazards (RAW): Solved primarily by forwarding/bypassing paths
4. Control hazards: Solved by branch prediction (95%+ accuracy in modern CPUs)
5. Stall cycles waste pipeline slots — every stall reduces throughput
6. Deeper pipelines have higher branch misprediction penalties
7. Modern CPUs use all techniques simultaneously: forwarding + prediction + reordering