This feature is exclusively available to Starter, Research, and Pro. Upgrade when you need this workflow, review pricing, or send a question before changing plans.
What you keep on Free
How do you estimate regression coefficients when your predictors are highly collinear or outnumber your observations?
Ordinary least squares breaks down when the design matrix is nearly singular. Multicollinearity inflates coefficient variances, making individual estimates unreliable even if the overall fit is good. Hoerl and Kennard (1970) proposed adding a penalty to the normal equations, shrinking coefficients toward zero and stabilizing the inversion. The resulting estimator, , is biased but can have dramatically lower mean squared error than OLS when collinearity is severe.
The mechanics are simple. The penalty added to the OLS objective function penalizes the sum of squared coefficients. This is equivalent to placing a Gaussian prior on the coefficient vector in a Bayesian regression. The penalty parameter controls the bias-variance tradeoff: recovers OLS, shrinks all coefficients to zero. Cross-validation or generalized cross-validation selects to minimize out-of-sample prediction error.
In macroeconomics, ridge regression appears whenever the predictor set is large relative to the sample. Nowcasting GDP from 100+ monthly indicators runs into a "fat" design matrix. Phillips-curve estimation with many candidate slack measures and their lags produces collinearity. International growth regressions with 60+ potential determinants (the Sala-i-Martin "I just ran two million regressions" problem) face the same issue. More recently, Plagborg-Moller and Wolf (2021) used ridge-regularized local projections to estimate impulse responses, showing that shrinkage can substantially tighten confidence bands at longer horizons.
The Federal Reserve Bank of New York's nowcasting model uses ridge-type shrinkage within its dynamic factor framework. The Bank of England's COMPASS suite applies regularization to large conditioning sets. Academic researchers at central banks routinely use ridge as a benchmark when comparing regularized versus factor-based forecasting methods. The estimator's computational simplicity---a single matrix inversion---makes it attractive when the model needs frequent re-estimation, as in real-time forecasting settings.
The input is a standard regression dataset: observations of a dependent variable and predictors . In macro applications, is typically a forecast target (next-quarter GDP growth, inflation) and collects a large set of candidate predictors (financial variables, survey indicators, labor market series, international data). The predictors are almost always standardized to unit variance before estimation so that the penalty treats all coefficients symmetrically.
The model solves , where is the penalty parameter. The closed-form solution is . Geometrically, OLS finds the coefficient vector that minimizes the residual sum of squares; ridge constrains that vector to lie within a sphere of radius for some that maps one-to-one to . The constraint sphere is smooth and convex, so the solution is unique and continuous in .
The penalty parameter is selected by -fold cross-validation (typically ) or generalized cross-validation (GCV), which provides an analytical approximation to leave-one-out CV. The GCV criterion is , where is the ridge hat matrix. The effective degrees of freedom decreases monotonically from (at ) to (as ), providing a continuous measure of model complexity.
The Federal Reserve Bank of New York's nowcasting framework uses regularized regressions (including ridge) when estimating bridge equations from large sets of monthly indicators. With 100+ candidate predictors released on staggered schedules, OLS on the full set is infeasible for most vintages. Ridge provides stable coefficient estimates that update smoothly as new data arrive, which is critical for a real-time forecasting system where erratic coefficient jumps would undermine credibility.
Phillips-curve estimation illustrates the collinearity problem ridge was built for. If you include six measures of economic slack (output gap, unemployment gap, capacity utilization gap, employment-to-population ratio, vacancy-unemployment ratio, underemployment rate) plus their first and second lags, you have 18 highly correlated predictors. OLS assigns wild, offsetting coefficients to near-identical series. Ridge pulls these coefficients toward each other (and toward zero), producing a composite slack measure implicitly. De Mol, Giannone, and Reichlin (2008) showed that ridge regression on a large predictor set can perform comparably to principal-components-based factor models for macro forecasting.
In academic applied work, ridge serves as a benchmark in forecast comparison exercises. Researchers evaluating new machine-learning methods for macro forecasting (random forests, neural networks, gradient boosting) almost always include ridge as a baseline because it is the simplest regularized method with a closed-form solution. When ridge beats a fancy model, it usually means the fancy model is overfitting. When a fancy model barely beats ridge, the nonlinearity gains are negligible.
Ridge fails when variable selection matters. It keeps every predictor in the model, shrinking weak ones toward zero but never eliminating them. For practitioners who need to identify which predictors drive the outcome---policy evaluation, structural analysis, model interpretability---ridge is the wrong tool. It also fails when the true model is very sparse: if only 5 of 100 predictors matter, LASSO or elastic net will outperform ridge by zeroing out the irrelevant 95. Finally, ridge treats all predictors as exchangeable through the isotropic penalty . If prior knowledge suggests that some groups of predictors should be penalized differently (e.g., financial variables less than survey variables), group-penalty methods like group LASSO or Bayesian hierarchical regression are more appropriate.
The vector of penalized regression coefficients, . Biased toward zero but lower variance than OLS.
Non-negative scalar controlling shrinkage intensity. Zero recovers OLS; infinity sends all coefficients to zero.
The matrix of predictor cross-products. Its eigenvalues determine collinearity severity; small eigenvalues signal near-singularity.
Trace of the hat matrix . Equals where are singular values of . Continuously interpolates between and .
Analytical approximation to leave-one-out CV error. Used to select without repeatedly refitting the model.
Measures collinearity of predictor with all others: . Values above 10 signal problematic collinearity that ridge was designed to address.
The conditional expectation is linear in for some unknown .
If violated: Ridge shrinks toward a wrong model. Residual plots and partial-response checks reveal nonlinearity.
The error terms are uncorrelated: for .
If violated: Cross-validation scores are invalid if errors are serially correlated (common in time-series data). Block cross-validation or time-series CV is required.
Errors have constant variance: for all .
If violated: GCV is inconsistent under heteroskedasticity. Use -fold CV instead, which does not require constant variance.
Predictors are standardized to zero mean and unit variance before applying the penalty.
If violated: Without standardization, the penalty penalizes coefficients on different scales unequally. A predictor measured in billions gets near-zero coefficient regardless of its importance.
The true coefficient vector is dense---most predictors contribute at least weakly to .
If violated: If the truth is sparse (only a few predictors matter), ridge keeps all of them in the model with small nonzero coefficients. LASSO, which sets some coefficients exactly to zero, is more appropriate under sparsity.
The selected satisfies as , so the ridge estimator converges to OLS in large samples.
If violated: If is fixed regardless of sample size, the bias does not vanish and the estimator is inconsistent. Cross-validation naturally selects smaller as grows.