Memoryless and Dynamic Systems

Introduction

The distinction between memoryless and dynamic systems is about whether a system's output at any given moment depends solely on the current input value or also on past (and possibly future) input values. A memoryless system is purely instantaneous — it has no concept of history or time evolution. A dynamic system, by contrast, retains information about previous inputs or states, giving it the ability to accumulate, average, or respond to temporal patterns.

This classification has profound practical implications. Memoryless systems are computationally simple and introduce no delay, but they cannot perform operations like filtering, integration, or pattern detection that inherently require comparing the present to the past. Dynamic systems can achieve sophisticated processing but require storage elements (capacitors, registers, memory) and introduce temporal dependencies that complicate analysis.

Formal Definitions

Memoryless (Static) System

A system is memoryless if the output at time $t$ (or index $n$) depends ONLY on the input at the same time $t$ (or $n$):

$$y(t) = f(x(t)) \quad \text{(continuous-time)}$$ $$y[n] = f(x[n]) \quad \text{(discrete-time)}$$

The function $f(\cdot)$ can be any mapping — linear or nonlinear — but it operates only on the instantaneous input value.

Dynamic (System with Memory)

A system has memory if the output at any time depends on input values at other times:

$$y(t) = g(x(t), x(t-\tau_1), x(t-\tau_2), \ldots) \quad \text{for some } \tau_i \neq 0$$

Or in general, the output depends on the input over some interval rather than at a single point.

Examples of Memoryless Systems

Resistor (Ohm\'s law): $v(t) = Ri(t)$ — voltage depends only on current at the same instant.

Ideal amplifier: $y(t) = Gx(t)$ — output is a scaled version of the present input.

Squarer: $y(t) = x^2(t)$ — nonlinear but memoryless.

Quantizer: $y[n] = Q(x[n])$ — maps each sample independently to the nearest level.

Ideal diode: $y(t) = \max(0, x(t))$ — half-wave rectification operates sample by sample.

Lookup table (combinational logic): Output determined entirely by current input state.

Examples of Dynamic Systems

Capacitor: $v(t) = \frac{1}{C}\int_{-\infty}^{t} i(\tau) d\tau$ — voltage depends on the entire past history of current.

Inductor: $v(t) = L\frac{di}{dt}$ — voltage depends on the rate of change (requires knowledge of nearby time values).

Moving average filter: $y[n] = \frac{1}{3}(x[n] + x[n-1] + x[n-2])$ — uses current and two past samples.

Accumulator: $y[n] = \sum_{k=-\infty}^{n} x[k]$ — running sum remembers all past inputs.

Delay element: $y[n] = x[n-1]$ — output is the previous input (requires one unit of memory).

Recursive filter: $y[n] = ay[n-1] + x[n]$ — output depends on its own past value.

The Role of Memory in Signal Processing

Memory enables fundamental operations that memoryless systems cannot perform:

Filtering: Separating desired frequency components from noise requires comparing the signal across multiple time instants to determine which frequencies are present. A memoryless system cannot distinguish a 100 Hz signal from a 1000 Hz signal at any single instant.

Integration/Accumulation: Computing a running sum or average inherently requires remembering past values.

Differentiation: Computing the rate of change requires comparing the current value with a recent past value.

Prediction: Estimating future signal values requires learning from past patterns — impossible without memory.

Detection: Detecting a temporal pattern (like a specific sequence of bits) requires comparing the current observation against stored templates.

Characterizing Memory Length

Dynamic systems differ in how much memory they use:

Finite memory: The output depends on a finite number of past inputs. An FIR filter of length $M$ has memory of $M-1$ samples. The system "forgets" inputs older than $M-1$ samples.

Infinite memory: The output depends on arbitrarily distant past inputs. IIR filters and integrators have infinite memory — theoretically, an infinitely old input still affects the current output (though its influence decays for stable systems).

Fading memory: A practical concept where the influence of past inputs decays over time. A stable IIR filter has fading memory: $|h[n]| \to 0$ as $n \to \infty$.

Implications for Implementation

Memoryless systems require:

• No storage elements
• No clock or timing (purely combinational)
• Zero processing delay (ideally)
• Simple hardware: resistors, amplifiers, lookup tables

Dynamic systems require:

• Storage elements: capacitors (analog), registers/RAM (digital)
• Clocking mechanisms (for synchronous digital systems)
• Initialization: initial conditions must be specified
• State management: internal state evolves over time

State-Space Interpretation

Dynamic systems can be described using state variables that capture the system\'s memory:

$$\dot{\mathbf{x}}(t) = A\mathbf{x}(t) + B\mathbf{u}(t) \quad \text{(state evolution)}$$ $$\mathbf{y}(t) = C\mathbf{x}(t) + D\mathbf{u}(t) \quad \text{(output equation)}$$

The state vector $\mathbf{x}(t)$ encodes all the memory (history) needed to determine future outputs. For a memoryless system, there is no state ($\mathbf{x} = 0$) and the output is $y = Du$ (direct feedthrough only).

Key Takeaways

• Memoryless: output at time $t$ depends only on input at time $t$ — no storage needed
• Dynamic: output depends on past (or future) inputs — requires memory elements
• Memoryless systems are instantaneous and simple; dynamic systems enable filtering and accumulation
• All practical filters, integrators, and differential equations represent dynamic systems
• Finite memory (FIR) vs infinite memory (IIR) determines the system\'s temporal reach
• State-space models formalize the concept of memory through state variables