Recursive Filters: SMA, EMA, Low‑Pass, and a Tiny Kalman

filters , signal-processing , math While exploring optimizers I fell down the rabbit hole of recursive filters. This post is a compact, practical tour of smoothers you can use when measurements are noisy but latency and compute are tight. We’ll keep the math minimal, the intuition high, and focus on when and why to use SMA, EMA/low‑pass, and a tiny 1D Kalman.

If you are here just for code, my Kaggle Notebook can be found here.

Recursive filters shine when compute and memory are scarce, or when data arrives as a stream and you must react immediately.

Recursive filters that we’ll use, all use constant‑size state. That’s why they’re common in embedded systems, robotics control loops, mobile sensor smoothing and telemetry.

Many libraries choose α \alpha α directly; our example consists of k k k data points and we define alpha as α = k − 1 k \alpha = \tfrac{k-1}{k} α = k k − 1 ​ to make notation easier to hold in our memory.

This behaves like a leaky integrator. Bigger α \alpha α forgets the past more slowly (smoother, more lag). The initial line takes a few iterations before it can get closer to the real mean value.

Windowed average over the last k k k samples:

Great at crushing noise, but it lags and needs a buffer of the last k k k points. Spikes are “diluted” equally across the window.

In discrete time, the classic low‑pass is algebraically the same as the EMA:

Different name, same form. You pick α \alpha α to trade off noise suppression versus responsiveness.

This is just scratching the surface of a Kalman Filter - a tiny 1D Kalman Update. This is my first attempt at building intuition before I can jump to an actual KF.

When you know your sensor noise ( R R R ) and process noise ( Q Q Q ), Kalman gives you an adaptive gain that automatically balances trust between the prediction and the measurement. For a constant‑value model ( A = H = 1 A=H=1 A = H = 1 ):

If measurements are noisy (large R R R ), K t K_t K t ​ shrinks and you trust the prior more. If the process is volatile (large Q Q Q ), P t − P_t^{-} P t − ​ grows and K t K_t K t ​ increases—trust the new measurement more.

SMA is simple and robust; EMA/low‑pass is the default for streaming; and a tiny Kalman filter adds principled adaptivity when you can estimate noise. Two first filters are easy to implement, Kalman though was simplied to its 1D version. An actual Kalman Filter is way more complicated and it should be the subject of my future learning. Hopefully there will be a text about it of a decent quality.