Image Processing

Introduction

Image processing extends the concepts of signals and systems from one dimension (time) to two dimensions (space). Every photograph you take on your phone is immediately processed by sophisticated algorithms — noise reduction, sharpening, color correction, and enhancement — all built on the same mathematical foundations of filtering, convolution, and frequency analysis that we study for 1D signals. Understanding image processing as a signals and systems engineer means seeing a photograph not as a picture, but as a 2D signal that can be analyzed, filtered, and transformed.

A digital image is fundamentally a 2D discrete signal $f[m, n]$ where $m$ and $n$ represent the row and column indices (spatial coordinates), and the value at each point represents intensity (for grayscale) or color information. For a grayscale image with 8-bit depth, pixel values range from 0 (black) to 255 (white).

2D Signals and Systems Framework

Extending 1D Concepts to 2D

Just as a 1D signal is a function of time $x(t)$, a 2D image is a function of two spatial variables $f(x, y)$ in the continuous domain or $f[m, n]$ in the discrete domain. All the key concepts translate directly:

• 1D convolution → 2D convolution: $y[m,n] = \sum_{k}\sum_{l} x[k,l] \cdot h[m-k, n-l]$
• 1D Fourier Transform → 2D DFT: $F[u,v] = \sum_{m=0}^{M-1}\sum_{n=0}^{N-1} f[m,n] e^{-j2\pi(um/M + vn/N)}$
• 1D filtering → 2D spatial filtering: Apply kernels (masks) across the image

The 2D Convolution

The output of a 2D LTI system is the 2D convolution of the input image with the system's impulse response (called the point spread function or PSF):

$$g[m, n] = f[m, n] * h[m, n] = \sum_{k=-\infty}^{\infty}\sum_{l=-\infty}^{\infty} f[k, l] \cdot h[m-k, n-l]$$

In practice, the kernel $h[m,n]$ is a small matrix (typically 3×3, 5×5, or 7×7) that slides across the image, computing a weighted sum at each position.

Spatial Domain Filtering

Smoothing (Low-Pass) Filters

Smoothing filters reduce noise by averaging neighboring pixels:

Mean filter (3×3): $$h = \frac{1}{9}\begin{bmatrix} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{bmatrix}$$

Gaussian filter: Weights follow a 2D Gaussian distribution: $$h[m, n] = \frac{1}{2\pi\sigma^2} e^{-(m^2+n^2)/(2\sigma^2)}$$

The Gaussian filter is preferred because its frequency response is also Gaussian (no ringing), and it is separable — a 2D Gaussian can be decomposed into two 1D Gaussians applied sequentially (row-wise then column-wise), reducing computation from $O(N^2 K^2)$ to $O(N^2 \cdot 2K)$.

Sharpening (High-Pass) Filters

Sharpening enhances edges by amplifying high spatial frequencies. The Laplacian operator approximates the second derivative:

$$\nabla^2 f = \frac{\partial^2 f}{\partial x^2} + \frac{\partial^2 f}{\partial y^2}$$

Discrete Laplacian kernel: $$h = \begin{bmatrix} 0 & -1 & 0 \\ -1 & 4 & -1 \\ 0 & -1 & 0 \end{bmatrix}$$

Unsharp masking is a practical sharpening technique: subtract a blurred version of the image from the original: $$g[m,n] = f[m,n] + \alpha(f[m,n] - f_{blurred}[m,n])$$

Edge Detection

Edges represent rapid intensity changes — they are detected using gradient operators.

Sobel operator: Computes horizontal and vertical gradients: $$G_x = \begin{bmatrix} -1 & 0 & 1 \\ -2 & 0 & 2 \\ -1 & 0 & 1 \end{bmatrix} * f, \quad G_y = \begin{bmatrix} -1 & -2 & -1 \\ 0 & 0 & 0 \\ 1 & 2 & 1 \end{bmatrix} * f$$

The gradient magnitude is: $|G| = \sqrt{G_x^2 + G_y^2}$

Canny edge detector (the standard approach):

1. Smooth with Gaussian filter (reduces noise)
2. Compute gradient magnitude and direction using Sobel
3. Apply non-maximum suppression (thin edges to single-pixel width)
4. Apply double thresholding and hysteresis (connect strong edges through weak ones)

Frequency Domain Image Processing

2D Discrete Fourier Transform

The 2D DFT decomposes an image into its spatial frequency components:

$$F[u, v] = \sum_{m=0}^{M-1}\sum_{n=0}^{N-1} f[m,n] \cdot e^{-j2\pi(um/M + vn/N)}$$

The inverse transform recovers the image:

$$f[m, n] = \frac{1}{MN}\sum_{u=0}^{M-1}\sum_{v=0}^{N-1} F[u,v] \cdot e^{j2\pi(um/M + vn/N)}$$

Low spatial frequencies (near the center of the spectrum) represent smooth, slowly varying regions. High spatial frequencies (far from center) represent edges and fine details.

Frequency Domain Filtering

The convolution theorem extends to 2D:

$$g[m,n] = f[m,n] * h[m,n] \quad \Leftrightarrow \quad G[u,v] = F[u,v] \cdot H[u,v]$$

This means: transform the image to frequency domain, multiply by the filter transfer function, and inverse transform. For large kernels, this is computationally more efficient than spatial convolution (using 2D FFT).

Ideal low-pass filter: $H[u,v] = 1$ for $\sqrt{u^2+v^2} \leq D_0$, and 0 otherwise. This causes ringing (Gibbs phenomenon).

Butterworth low-pass filter: $H[u,v] = \frac{1}{1 + (D/D_0)^{2n}}$ — smoother transition, less ringing.

Gaussian low-pass filter: $H[u,v] = e^{-D^2/(2D_0^2)}$ — no ringing at all, optimal smoothness.

Image Enhancement Techniques

Histogram Equalization

This technique improves contrast by redistributing pixel intensities to span the full dynamic range. For an image with $L$ gray levels, the transformation is:

$$s_k = (L-1) \sum_{j=0}^{k} p(r_j) = (L-1) \cdot \text{CDF}(k)$$

where $p(r_j)$ is the probability of intensity level $j$. This maps the cumulative distribution function (CDF) of the input to a uniform distribution, maximizing image contrast.

Homomorphic Filtering

Images can be modeled as the product of illumination and reflectance:

$$f(x,y) = i(x,y) \cdot r(x,y)$$

Taking the logarithm converts this to a sum, allowing separate filtering. A homomorphic filter attenuates low frequencies (illumination variations) while enhancing high frequencies (reflectance/edges), reducing uneven lighting effects.

Applications

• Medical imaging: CT/MRI reconstruction, X-ray enhancement, tumor detection
• Remote sensing: Satellite image analysis, terrain classification
• Computer vision: Object detection, facial recognition, autonomous driving
• Photography: HDR imaging, computational photography, noise reduction in smartphones
• Document processing: OCR preprocessing, binarization, deskewing

Key Takeaways

• Images are 2D discrete signals; all 1D signal processing concepts extend to 2D with appropriate modifications
• 2D convolution with small kernels implements spatial filters for smoothing, sharpening, and edge detection
• The 2D DFT enables frequency-domain analysis and filtering of images using the convolution theorem
• Edge detection (Sobel, Canny) uses gradient estimation — a form of high-pass filtering
• Gaussian filters are preferred for smoothing because they are separable and have no ringing
• The 2D FFT makes large-kernel filtering computationally efficient in the frequency domain