Floating Point Arithmetic

Introduction

Floating-point arithmetic is fundamentally different from integer arithmetic because each number consists of two parts — an exponent and a significand (mantissa) — that must be managed together during every operation. This makes floating-point hardware significantly more complex than integer hardware. A simple addition that takes one cycle for integers requires multiple stages for floating-point: exponent comparison, significand alignment, addition, normalization, and rounding.

Modern processors contain dedicated Floating-Point Units (FPUs) with pipelined execution stages, allowing them to achieve throughput of one floating-point operation per cycle despite each individual operation having a multi-cycle latency (typically 3-5 cycles for addition, 4-6 cycles for multiplication).

Floating-Point Addition and Subtraction

Adding two floating-point numbers is the most complex of the basic operations because the significands must be aligned before they can be added. The algorithm proceeds in five stages:

Step 1: Compare Exponents

Determine which operand has the larger exponent by computing the difference d = |E1 - E2|. The number with the smaller exponent must be adjusted.

Step 2: Align Significands

Shift the significand of the smaller number right by d positions. This makes the exponents equal. For example, adding 1.5 × 10³ and 2.3 × 10¹: shift 2.3 to 0.023 so both have exponent 3.

Step 3: Add or Subtract Significands

With aligned exponents, add or subtract the significands as integers. The result may require more bits than the significand width.

Step 4: Normalize the Result

The result must be in normalized form (1.xxx... × 2^E for IEEE 754). If the result has a carry-out (like 10.xxx), shift right by 1 and increment the exponent. If leading zeros appear (like 0.001xxx), shift left until the leading 1 is in the correct position and decrement the exponent accordingly.

Step 5: Round the Result

Since the right-shift in step 2 and the significand addition may produce more bits than can be stored, the result must be rounded to fit the available precision. After rounding, re-normalize if necessary (rounding can cause a carry).

Worked Example

Add A = 1.101 × 2³ and B = 1.001 × 2¹ (using 4-bit significands):

1. Exponent difference: 3 - 1 = 2
2. Shift B right by 2: B becomes 0.01001 × 2³
3. Add significands: 1.101 + 0.010 = 1.111 (truncating extra bits)
4. Result: 1.111 × 2³ (already normalized)
5. No rounding needed (fits in 4 bits)

Floating-Point Multiplication

Multiplication is actually simpler than addition for floating-point numbers because no alignment step is needed. The algorithm:

Step 1: Add Exponents

The product's exponent = E1 + E2 - Bias. We subtract the bias once because both operands have the bias added, and the product should have it only once. For IEEE 754 single precision (bias = 127): E_result = E1 + E2 - 127.

Step 2: Multiply Significands

Multiply the two significands as unsigned integers. For two numbers of the form 1.f1 × 1.f2, the product is in the range [1.0, 4.0) — meaning it is either 1.xxx or 1x.xxx (at most one bit of overflow).

Step 3: Normalize

If the product significand is ≥ 2.0 (i.e., the form is 1x.xxx), shift right by 1 and increment the exponent.

Step 4: Round

Round to fit the available precision bits and check for overflow/underflow of the exponent.

Example

Multiply 1.5 × 2² by 1.25 × 2³:

• Significands: 1.100 × 1.010 = 10.001100 (shift right → 1.0001100, exponent +1)
• Exponents: 2 + 3 + 1(normalization) = 6
• Result: 1.0001100 × 2⁶ (round to available bits)
• Check: 1.5 × 4 × 1.25 × 8 = 6 × 10 = 60. In binary: 1.0001100 × 2⁶ ≈ 60. Correct order of magnitude.

Floating-Point Division

Division follows a similar pattern to multiplication:

Step 1: Subtract Exponents

E_result = E1 - E2 + Bias. The bias is added back because subtracting two biased exponents removes the bias entirely.

Step 2: Divide Significands

Divide the dividend significand by the divisor significand. The result is in the range (0.5, 2.0).

Step 3: Normalize

If the result is less than 1.0, shift left and decrement the exponent.

Step 4: Round

Round to the target precision.

Division is the slowest floating-point operation, typically taking 10-20 cycles even with advanced algorithms like SRT or Newton-Raphson iterative approximation.

Rounding Modes

IEEE 754 specifies four rounding modes, and the FPU hardware must support all of them:

1. Round to Nearest Even (default): Rounds to the nearest representable value. If exactly halfway, rounds to make the last bit even. This minimizes statistical bias.
2. Round toward +∞ (ceiling): Always rounds up toward positive infinity.
3. Round toward -∞ (floor): Always rounds down toward negative infinity.
4. Round toward Zero (truncation): Simply discards extra bits.

Hardware implements rounding using guard bits, round bit, and sticky bit — three extra bits maintained during computation to determine the correct rounding decision.

Special Cases

The FPU must handle several special cases defined by IEEE 754:

• Infinity: Operations involving ∞ follow mathematical rules (∞ + x = ∞, ∞ × positive = ∞)
• NaN (Not a Number): Results from undefined operations (0/0, ∞-∞, √negative). Any operation involving NaN produces NaN.
• Denormalized numbers: Very small numbers below normal range are represented with a leading 0 instead of 1, allowing gradual underflow.
• Zero: Both +0 and -0 exist; the FPU must handle both correctly.

FPU Pipeline Architecture

Modern FPUs pipeline the floating-point operations so that while one addition is in the normalization stage, another can be in the alignment stage, and a third can be in the exponent comparison stage. A typical FPU addition pipeline:

With 4-stage pipelining, the FPU achieves 1 result per cycle throughput despite 4-cycle latency for each individual operation.

Key Takeaways

1. Floating-point addition requires exponent alignment before significands can be added — making it multi-stage
2. Floating-point multiplication adds exponents and multiplies significands — simpler than addition
3. Normalization ensures results maintain the standard 1.xxx form required by IEEE 754
4. Rounding uses guard, round, and sticky bits to correctly round results to the target precision
5. Special values (infinity, NaN, denormals) require dedicated detection and handling logic
6. Modern FPUs pipeline operations to achieve one-result-per-cycle throughput despite multi-cycle latency
7. Floating-point division is the slowest operation, often using iterative methods like Newton-Raphson