Floating Point Representation

Introduction

Integers are great for counting, but what about numbers like 3.14159, 0.001, or 6.022 × 10²³? Fixed-point representation can handle fractions, but it wastes bits when dealing with very large or very small numbers. The solution is floating-point representation — essentially scientific notation in binary. This is how computers handle real numbers, and understanding it explains why floating-point math sometimes gives surprising results.

The Need for Floating Point

Consider representing these numbers in a 32-bit fixed-point format:

• 0.000000001 (very small) — needs many bits after the decimal point
• 3,000,000,000 (very large) — needs many bits before the decimal point

You can't efficiently represent both with the same fixed-point format. Floating point solves this by dynamically adjusting where the "decimal point" sits.

Scientific Notation: The Analog

In decimal scientific notation:

The format is: ±M × 10^E

• Sign: positive or negative
• Mantissa (M): the significant digits (normalized: 1 ≤ M < 10)
• Exponent (E): where to place the decimal point

Floating point does the same thing in binary: ±M × 2^E

Binary Floating Point Format

A floating-point number has three parts stored in binary:

Sign Bit

• 0 = positive
• 1 = negative

Exponent

Determines the magnitude (how far to shift the binary point). Stored with a bias so that both positive and negative exponents can be represented without a separate sign bit.

Biased exponent = Actual exponent + Bias Bias = 2^(e-1) - 1

For 8-bit exponent: Bias = 127 So actual exponent of +5 is stored as 132 (5+127) And actual exponent of -3 is stored as 124 (-3+127)

Mantissa (Significand)

The fractional part of the normalized number. In binary normalization, we write the number as:

Since the leading 1 is always there (for normalized numbers), we don't need to store it — this is called the hidden bit or implicit leading 1. We only store the fractional part (xxxxx), gaining one extra bit of precision for free!

Normalization

A binary floating-point number is normalized when it's in the form:

Example: Normalize 101.11₂

Move the binary point left 2 positions → exponent = 2

Example: Normalize 0.00101₂

Move the binary point right 3 positions → exponent = -3

Step-by-Step Conversion: Decimal to Floating Point

Let's convert -6.75 to a 32-bit floating-point representation (single precision):

Step 1: Convert to binary

• 6 = 110₂
• 0.75 = 0.11₂ (0.75 × 2 = 1.5 → 1; 0.5 × 2 = 1.0 → 1)
• So 6.75 = 110.11₂

Step 2: Normalize

• 110.11 = 1.1011 × 2²
• Mantissa (fraction) = 1011
• Exponent = 2

Step 3: Apply bias

• Biased exponent = 2 + 127 = 129 = 10000001₂

Step 4: Assemble

Step-by-Step Conversion: Floating Point to Decimal

Given: 0 10000010 10100000000000000000000

Step 1: Extract components

• Sign = 0 (positive)
• Exponent = 10000010₂ = 130
• Mantissa = 10100000000000000000000

Step 2: Calculate actual exponent

• Actual exponent = 130 - 127 = 3

Step 3: Reconstruct number

• Add the hidden bit: 1.10100000...
• Multiply by 2³: shift point right 3 → 1101.0
• Value = 1101.0₂ = 8+4+1 = 13.0₁₀

Result: +13.0

Special Values

Floating point reserves certain bit patterns for special cases:

Denormalized Numbers

When the exponent is all zeros, the hidden bit is 0 (not 1), allowing representation of very small numbers near zero. This prevents a sudden "gap" between zero and the smallest normal number.

NaN (Not a Number)

Results from undefined operations like 0/0 or √(-1). NaN has special propagation rules — any arithmetic with NaN produces NaN.

Precision and Rounding Errors

Floating point cannot represent all real numbers exactly. Consider 0.1 in binary:

Since we have finite mantissa bits, this gets truncated, introducing a tiny error. This is why in programming:

This isn't a bug — it's an inherent limitation of binary floating-point representation.

Key Takeaways

1. Floating point uses sign + exponent + mantissa format, analogous to scientific notation
2. Normalization ensures the number is in 1.xxx × 2^E form
3. The hidden bit gives one extra bit of free precision
4. The biased exponent allows representing both positive and negative exponents without a sign bit
5. Special encodings handle zero, infinity, NaN, and denormalized numbers
6. Floating point has inherent precision limitations — not all decimal fractions can be represented exactly
7. Understanding floating point explains many "bugs" in numerical computation