Pipeline Performance

Introduction

How much does pipelining actually improve performance? What's the theoretical maximum, and why can't we achieve it in practice? This chapter provides the mathematical framework for analyzing pipeline performance, calculating speedup, understanding efficiency, and quantifying the impact of stalls and hazards.

Speedup Formula

Ideal Speedup

For a k-stage pipeline executing n instructions:

Without pipeline: Time = n × k × τ (where τ is the time per stage) With pipeline: Time = (k + n - 1) × τ

As n → ∞: Speedup → k (the number of pipeline stages)

For a 5-stage pipeline with 1000 instructions:

Nearly the ideal 5× for large programs!

Real-World Speedup (with Stalls)

In practice, stalls reduce performance:

If a 5-stage pipeline has an average of 0.5 stall cycles per instruction:

Pipeline Throughput

Throughput = instructions completed per unit time:

Ideal throughput: 1 instruction per cycle (IPC = 1) Actual throughput: IPC = 1/CPI = 1/(1 + stalls)

Sources of Performance Loss

1. Pipeline Stalls (from hazards)

Typical breakdown:

• Data hazards (load-use): 0.05-0.15 stalls/instruction
• Control hazards (branches): 0.05-0.30 stalls/instruction (depends on prediction accuracy)
• Structural hazards: ~0 in well-designed pipelines

2. Unbalanced Stages

If stages have different latencies, the clock is limited by the slowest:

If MEM takes 300ps but other stages take 200ps:

3. Pipeline Register Overhead

Pipeline registers add latency (typically 10-30ps per register):

With 5 stages at 200ps each and 20ps register delay:

Branch Prediction Impact

Example: 20% branches, 5% misprediction rate, 3-cycle penalty, 10% load-use:

Pipeline Efficiency

Or expressed as utilization:

Deep Pipeline Analysis

Deeper Pipeline Pros

• Higher clock frequency (shorter stages)
• Higher throughput potential

Deeper Pipeline Cons

• Higher branch misprediction penalty
• More pipeline stages to flush
• More forwarding paths needed
• Higher power consumption (more pipeline registers)
• Diminishing returns from shorter stages (wire delays don't scale)

Optimal Pipeline Depth

Studies suggest 10-15 stages is optimal for most workloads. Beyond that, the penalty growth outweighs the frequency benefit:

Performance Comparison Example

Key Takeaways

1. Ideal pipeline speedup equals the number of stages (k), achievable only with zero stalls
2. Real speedup = k / (1 + stalls_per_instruction), always less than ideal
3. CPI_pipelined = 1 + stalls; IPC = 1/(1+stalls)
4. Branch mispredictions dominate performance loss in modern pipelines
5. Deeper pipelines offer higher clock speed but suffer greater misprediction penalties
6. Pipeline register overhead and unbalanced stages reduce efficiency
7. Optimal pipeline depth balances clock speed gains against increased hazard penalties