Pipeline Introduction

Introduction

Pipelining is the single most important technique for improving CPU performance. The idea is beautifully simple: instead of waiting for one instruction to completely finish before starting the next, overlap the execution of multiple instructions — like an assembly line in a factory. While instruction #1 is executing, instruction #2 is being decoded, and instruction #3 is being fetched. This doesn't make any single instruction faster, but it dramatically increases throughput — the number of instructions completed per unit time.

The Assembly Line Analogy

Think of a car assembly line with 5 stations:

1. Build frame (15 minutes)
2. Install engine (15 minutes)
3. Add body panels (15 minutes)
4. Paint (15 minutes)
5. Final inspection (15 minutes)

Without pipelining (one car at a time):

• One car takes 75 minutes (5 × 15)
• Output: 1 car every 75 minutes

With pipelining (assembly line):

• Still 75 minutes for the FIRST car
• But after that, a new car finishes every 15 minutes!
• Output: 1 car every 15 minutes (5× improvement)

The same logic applies to CPU instructions.

The Five-Stage Pipeline

The classic CPU pipeline mirrors the five execution stages:

Pipeline Operation Visualization

After the pipeline is "full" (cycle 5 onward), one instruction completes every cycle!

Performance Benefits

Without Pipeline (Sequential)

• Each instruction: 5 cycles
• 100 instructions: 500 cycles

With Pipeline (Ideal)

• First instruction: 5 cycles
• Each additional: 1 cycle
• 100 instructions: 5 + 99 = 104 cycles

Speedup

Theoretical maximum speedup = number of pipeline stages (5× for a 5-stage pipeline)

Pipeline Terminology

Pipeline Registers

Between each stage, pipeline registers store intermediate results:

These registers capture all information needed for subsequent stages:

• IF/ID register: holds instruction bits and PC+4
• ID/EX register: holds decoded values, register data, control signals
• EX/MEM register: holds ALU result, branch target, write data
• MEM/WB register: holds memory data or ALU result for write back

Pipeline Timing

Critical insight: The clock cycle time is determined by the slowest stage:

If memory access takes 200ps and other stages take 100ps, the cycle time must be 200ps. This is why balanced stages are important — if one stage is much slower than others, it limits the entire pipeline.

Why Doesn't Pipelining Give 5× Speedup in Practice?

Several factors reduce real performance below the theoretical maximum:

1. Pipeline hazards: Situations requiring stalls (data dependencies, branches)
2. Unbalanced stages: Some stages take longer than others
3. Pipeline overhead: Pipeline registers add latency
4. Start-up/drain: Pipeline must fill at start and drain at end of programs

Real speedup is typically 60-80% of theoretical maximum.

Pipeline Depth Trade-offs

Deeper Pipeline (More Stages)

• Higher clock frequency (each stage does less work)
• More instructions in flight
• BUT: more hazards, higher branch misprediction penalty
• Example: Pentium 4 had 20+ stages for high clock speeds

Shallower Pipeline (Fewer Stages)

• Lower clock frequency
• Fewer hazards, lower misprediction penalty
• BUT: each stage does more work (longer cycle time)
• Example: ARM Cortex-A7 has 8 stages for efficiency

Key Takeaways

1. Pipelining overlaps instruction execution stages for higher throughput
2. It doesn't speed up individual instructions — it speeds up the instruction stream
3. Maximum theoretical speedup equals the number of pipeline stages
4. Pipeline registers between stages store intermediate results
5. Clock speed is limited by the slowest pipeline stage
6. Pipeline hazards (data, control, structural) prevent achieving ideal speedup
7. Pipeline depth is a trade-off: deeper = faster clock but more hazard penalties