Sampling Theorem (Shannon-Nyquist)

Introduction

The Sampling Theorem is arguably the single most important result in all of signal processing. It answers a question that might seem impossible at first: can you capture all the information in a continuous signal using only discrete samples taken at isolated points? The theorem says yes — provided two conditions are met. The signal must be bandlimited (it contains no frequencies above some maximum $f_{max}$), and the samples must be taken at a rate exceeding $2f_{max}$ samples per second.

This result, established independently by Nyquist, Shannon, Whittaker, and Kotelnikov, is what makes the entire digital revolution possible. Without it, we could not be confident that digitizing a signal preserves its information content. With it, we know exactly what sampling rate is needed and exactly how to recover the original continuous signal from its samples.

Formal Statement

The Nyquist-Shannon Sampling Theorem: A bandlimited signal $x(t)$ with $X(j\omega) = 0$ for $|\omega| > \omega_{max}$ is completely determined by its samples $x(nT_s)$ if:

$$f_s > 2f_{max} \quad \text{or equivalently} \quad T_s < \frac{1}{2f_{max}}$$

The original signal can be perfectly reconstructed using:

$$x(t) = \sum_{n=-\infty}^{\infty} x(nT_s) \cdot \text{sinc}\left(\frac{t - nT_s}{T_s}\right)$$

Proof via Frequency Domain Analysis

Step 1: Sampling as Multiplication

Ideal sampling multiplies the continuous signal by an impulse train:

$$x_s(t) = x(t) \cdot p(t) = x(t) \cdot \sum_{n=-\infty}^{\infty}\delta(t - nT_s)$$

This gives: $x_s(t) = \sum_{n=-\infty}^{\infty} x(nT_s)\delta(t - nT_s)$

Step 2: Spectrum of the Sampled Signal

The Fourier Transform of the impulse train is another impulse train: $$P(j\omega) = \frac{2\pi}{T_s}\sum_{k=-\infty}^{\infty}\delta(\omega - k\omega_s)$$

Using the multiplication-convolution duality: $$X_s(j\omega) = \frac{1}{2\pi}X(j\omega) * P(j\omega) = \frac{1}{T_s}\sum_{k=-\infty}^{\infty}X(j(\omega - k\omega_s))$$

The sampled signal's spectrum consists of the original spectrum $X(j\omega)$ replicated at every integer multiple of the sampling frequency $\omega_s$.

Step 3: Non-Overlapping Condition

The spectral replicas do not overlap if and only if: $$\omega_s - \omega_{max} > \omega_{max} \implies \omega_s > 2\omega_{max}$$

When this condition is satisfied, the baseband copy (centered at $\omega = 0$) is completely isolated from its neighbors.

Step 4: Reconstruction via Low-Pass Filter

An ideal low-pass filter with cutoff $\omega_c$ where $\omega_{max} < \omega_c < \omega_s - \omega_{max}$ extracts only the baseband replica:

$$H(j\omega) = \begin{cases} T_s, & |\omega| < \omega_c \\ 0, & |\omega| > \omega_c \end{cases}$$

The filter output is exactly $X(j\omega)$, so $x(t)$ is perfectly recovered.

What Happens When the Theorem is Violated

If $f_s < 2f_{max}$ (undersampling), the spectral replicas overlap. The overlapping regions add together, creating frequency components that are sums of the original spectrum and its shifted copies. This phenomenon is called aliasing, and it is irreversible — no amount of filtering can separate the overlapped components after the fact.

Conditions and Assumptions

The Sampling Theorem relies on several assumptions that are important to understand:

1. Bandlimited signal: $X(j\omega) = 0$ for $|\omega| > \omega_{max}$. In practice, no physical signal is strictly bandlimited (a time-limited signal cannot be bandlimited, and vice versa). However, practical signals can be made effectively bandlimited by applying an anti-aliasing filter before sampling.

1. Uniform sampling: Samples are taken at exactly equal intervals $T_s$. Non-uniform sampling is possible but requires different reconstruction formulas.

1. Infinite number of samples: The reconstruction formula sums over all samples from $-\infty$ to $+\infty$. In practice, finite truncation introduces small errors.

1. Ideal impulse sampling: Samples are instantaneous (zero-width). Real ADCs have finite aperture times, which introduces a slight low-pass filtering effect.

Practical Implications

Anti-Aliasing Filter Design

Before any ADC, an analog low-pass filter must ensure that no energy exists above $f_s/2$. The filter requirements are:

• Passband: 0 to $f_{max}$ (flat response desired)
• Stopband: $f_s/2$ and above (sufficient attenuation)
• Transition band: $f_{max}$ to $f_s/2$ (filter rolloff region)

A wider gap between $f_{max}$ and $f_s/2$ (more oversampling) allows a more gradual filter rolloff, making the filter cheaper and easier to build.

Choosing the Sampling Rate

The sampling rate is chosen based on:

• Signal bandwidth (must exceed $2f_{max}$)
• Available filter technology (determines needed guard band)
• Storage/processing constraints (higher rates mean more data)
• Application requirements (latency, real-time processing)

Standard Sampling Rates

The Time-Limited vs. Bandlimited Dilemma

A mathematical subtlety: a signal that is strictly time-limited (zero outside some interval) cannot be strictly bandlimited, and vice versa. This is proven by the Paley-Wiener theorem. Therefore, no real-world signal perfectly satisfies the theorem's bandlimited assumption.

In practice, this is resolved by:

1. Treating the signal as "effectively bandlimited" (energy above $f_{max}$ is negligible)
2. Using an anti-aliasing filter to enforce the bandwidth limit
3. Accepting the minuscule error from cutting off the spectral tails

Key Takeaways

• A bandlimited signal ($f_{max}$) is perfectly recoverable from samples taken at rate $f_s > 2f_{max}$
• Sampling replicates the spectrum at multiples of $f_s$ — non-overlapping replicas mean no information loss
• Reconstruction uses an ideal low-pass filter (equivalent to sinc interpolation)
• Violating the theorem ($f_s < 2f_{max}$) causes aliasing — irreversible corruption
• Anti-aliasing filters are mandatory before sampling in any practical system
• The theorem bridges the continuous and discrete worlds, enabling all of digital signal processing