Rudolf Kalman published 'A New Approach to Linear Filtering and Prediction Problems' in 1960. The result was immediate and far-reaching: given a linear state-space model with Gaussian disturbances, the Kalman filter computes the exact conditional mean and covariance of the latent state at each point in time, using only the previous state estimate and the current observation. No other linear estimator achieves lower mean-squared error. The filter was adopted first by aerospace engineers (Apollo navigation systems), then by electrical engineers, and by the late 1980s it had become central to macroeconometrics through the work of Harvey (1989) and Hamilton (1994).

The state-space representation splits the world into what you observe (measurement equation) and what you model (transition equation). The transition equation describes how the latent state evolves over time - trend growth, cyclical dynamics, time-varying parameters, factor loadings. The measurement equation links the latent state to observable data, with measurement noise. The Kalman filter takes this two-equation system and produces the prediction step (project the state forward using the transition equation) and the update step (revise the projection when new data arrive). The update step is the key: it weights the model's prediction against the data observation using the Kalman gain, which is optimal in the MSE sense.

In macroeconomics, the Kalman filter serves three roles. First, it estimates unobservable states: the output gap, the natural rate of interest, underlying inflation trends. Second, it evaluates the likelihood function of any model that can be written in state-space form, enabling maximum likelihood estimation of structural parameters. Third, it handles missing data and mixed-frequency observations naturally - if an observation is missing at time t, the update step is skipped and the prediction step carries the state forward under increased uncertainty.

The filter's optimality is conditional on linearity and Gaussianity. When the state-space model is nonlinear (e.g., DSGE models with occasionally binding constraints), the extended Kalman filter (EKF) or unscented Kalman filter (UKF) provides approximations. When disturbances are non-Gaussian, the particle filter replaces the Kalman filter with sequential Monte Carlo. But for the large class of linear Gaussian models that dominate applied macroeconometrics - UCMs, dynamic factor models, TVP-VARs, nowcasting systems - the Kalman filter is exact and optimal.

The Kalman filter requires two inputs: a fully specified state-space model (system matrices F, H, Q, R and initial conditions) and a sequence of observations $y_1$, ..., $y_T$. The system matrices can be fixed or time-varying. The state vector alpha_t has dimension m, which can be much larger than the observation dimension n. Deterministic inputs (intercepts, exogenous variables) enter through both equations.

The recursive algorithm has two phases per time step. The prediction step uses the transition equation to project the state and its covariance one step ahead: alpha_{t|t-1} = F * alpha_{t-1|t-1} and $P_{t|t-1} = F * P_{t-1|t-1} * F$' + Q. The update step incorporates the new observation $y_t$: compute the innovation $v_t = y_t - H *$ alpha_{t|t-1}, the innovation variance $S_t = H * P_{t|t-1} * H$' + R, the Kalman gain $K_t = P_{t|t-1} * H$' * $S_t^{-1}$, and then update: alpha_{t|t} = alpha_{t|t-1} + $K_t * v_t$ and $P_{t|t} =$ (I - $K_t * H) * P_{t|t-1}$. The Kalman gain determines how much weight to give the data versus the model prediction.

The filter's outputs include the filtered state alpha_{t|t} (using data up to time t), the predicted state alpha_{t|t-1} (using data up to t-1), the innovation sequence $v_t$ (prediction errors), and the innovation variances $S_t$. The log-likelihood is computed as a byproduct: sum of log-densities of $v_t$ evaluated under $N(0, S_t)$. For smoothing (estimating alpha_t using the full sample 1..T), the Rauch-Tung-Striebel (RTS) backward smoother runs after the forward filter pass.

The Federal Reserve Board's FRB/US model, the European Central Bank's area-wide model, and virtually every central bank's nowcasting system use the Kalman filter to estimate latent states - the output gap, the natural rate of unemployment (NAIRU), and the neutral real interest rate (r*). Laubach and Williams (2003) estimated r* using a two-sided Kalman smoother that became a benchmark for monetary policy discussions.

Dynamic factor models for coincident economic indicators (Stock and Watson 1991, 2002) run the Kalman filter on a state-space representation where a small number of latent factors drive a large panel of observed series. The Federal Reserve Bank of New York's nowcasting model (Bok et al. 2018) is essentially a dynamic factor model estimated via the Kalman filter with news decomposition.

The Kalman filter requires a correctly specified linear Gaussian model. When the system is nonlinear (DSGE models with zero-lower-bound constraints), the EKF or particle filter is needed. When disturbances have fat tails (financial returns during crises), the Kalman filter underweights extreme observations and the filtered state lags reality. With very high-dimensional state vectors (m > 100), the Kalman filter's $O(m^3)$ matrix inversions become expensive per time step, motivating square-root filters or ensemble methods.

Extensions include the square-root filter (numerically stable for ill-conditioned P matrices), the information filter (propagates $P^{-1}$ instead of P, useful when some observations are very precise), the steady-state Kalman filter (pre-computes the converged gain K for computational speed), and the augmented Kalman filter for correlated state and measurement disturbances.