Convolution Introduction

Introduction

Convolution is the mathematical operation that answers the fundamental question of LTI system theory: given an input signal and a system characterized by its impulse response, what is the output? The convolution operation combines two functions to produce a third function that represents the "amount of overlap" as one function slides across the other. It is the most important operation in linear signal processing, appearing in filtering, probability theory, image processing, and virtually every branch of applied mathematics.

The concept emerges naturally from two physical principles: any input signal can be decomposed into a collection of weighted, shifted impulses, and an LTI system\'s response to each impulse is a weighted, shifted copy of the impulse response. Superposition (linearity) then tells us to add all these individual responses together — and that sum IS the convolution integral.

Motivation: From Impulse Response to General Response

Suppose we know the impulse response $h(t)$ of an LTI system. How do we find the output for an arbitrary input $x(t)$?

Step 1: Represent the input as a sum of impulses. In the continuous case: $$x(t) = \int_{-\infty}^{\infty} x(\tau)\delta(t - \tau)d\tau$$

This says: "$x(t)$ is built from infinitesimal impulses at every time $\tau$, each weighted by $x(\tau)$."

Step 2: By time-invariance, the response to $\delta(t-\tau)$ is $h(t-\tau)$ (shifted impulse response).

Step 3: By linearity, the response to $x(\tau)\delta(t-\tau)$ is $x(\tau)h(t-\tau)$.

Step 4: By superposition, the total output is the sum (integral) of all individual responses: $$y(t) = \int_{-\infty}^{\infty} x(\tau)h(t-\tau)d\tau$$

This is the convolution integral. It follows inevitably from the LTI properties — it is not an arbitrary definition but a necessary consequence of linearity and time-invariance.

Definition

Continuous-time convolution: $$y(t) = x(t) * h(t) = \int_{-\infty}^{\infty} x(\tau)h(t-\tau)d\tau$$

Discrete-time convolution: $$y[n] = x[n] * h[n] = \sum_{k=-\infty}^{\infty} x[k]h[n-k]$$

The asterisk $*$ denotes convolution (not multiplication).

Physical Interpretation

Convolution has several complementary physical interpretations:

Weighted running average: The output at time $t$ is a weighted average of the input, where the weights come from the (reversed) impulse response. Recent inputs are weighted by $h(0)$, slightly older inputs by $h(1)$, etc.

Memory and response: The impulse response $h(t)$ encodes how the system "remembers" past inputs. A long impulse response means the system has a long memory — distant past inputs still influence the current output. A short impulse response means quick response with little memory.

Smoothing and spreading: Convolution always tends to spread or smooth signals. Convolving a sharp spike with a broad function produces a broad output. This is why convolution is used for smoothing (low-pass filtering) and why it appears in diffusion and blur models.

The "Flip and Slide" Interpretation

The convolution integral at time $t$ involves:

1. Flip $h(\tau)$ to get $h(-\tau)$ — this reverses the impulse response
2. Slide to position $t$: $h(t-\tau)$ — move the reversed function to be centered at $t$
3. Multiply $x(\tau) \cdot h(t-\tau)$ — compute the pointwise product
4. Integrate — sum the product over all $\tau$

As $t$ varies, the flipped impulse response slides across the input signal, and the integral measures the "overlap" at each position. The output $y(t)$ traces out this overlap as a function of $t$.

Why "Convolution" and Not Just "Multiplication"?

Convolution and multiplication are very different operations:

• Multiplication: pointwise product at each instant, $y(t) = x(t) \cdot h(t)$
• Convolution: integrated product with one function reversed and sliding

However, they are related by the Fourier Transform:

• Convolution in time ↔ Multiplication in frequency
• Multiplication in time ↔ Convolution in frequency (scaled)

This duality is what makes frequency-domain analysis so powerful: the complicated integral becomes simple multiplication after transformation.

Convolution in Other Contexts

Probability: If $X$ and $Y$ are independent random variables with PDFs $f_X$ and $f_Y$, the PDF of $Z = X + Y$ is $f_Z = f_X * f_Y$ (convolution of densities).

Image processing: 2D convolution with a kernel blurs, sharpens, or detects edges in images.

Neural networks: Convolutional neural networks (CNNs) apply learned convolution kernels to extract features from images.

Polynomial multiplication: Convolving coefficient sequences multiplies the corresponding polynomials.

Overview of Computation Methods

Key Takeaways

• Convolution computes LTI system output: $y = x * h$ (input convolved with impulse response)
• It arises from combining impulse decomposition with linearity and time-invariance
• Physical interpretation: weighted running average where weights come from the impulse response
• Convolution in time equals multiplication in frequency — the foundation of frequency-domain analysis
• "Flip, slide, multiply, integrate" is the graphical procedure for evaluation
• Convolution appears in probability, image processing, neural networks, and polynomial algebra