Data-Driven Models
Data-Driven Models
Empirical forecasting models · Model guide
Regularized regression that blends LASSO selection with ridge shrinkage for correlated predictor blocks.
How do you select variables from a large set of correlated predictors without arbitrarily dropping members of correlated groups?
LASSO (Tibshirani, 1996) performs variable selection by imposing an L1 penalty on regression coefficients, shrinking some exactly to zero. Ridge regression (Hoerl and Kennard, 1970) imposes an L2 penalty that shrinks all coefficients toward zero but never eliminates any. Each method has a blind spot. LASSO is unstable when predictors are correlated: it arbitrarily picks one from a correlated cluster and zeros the rest, and which one survives can flip with small data perturbations. Ridge cannot produce sparse models at all. Zou and Hastie (2005) proposed the elastic net, which adds both penalties simultaneously: . The mixing parameter interpolates between pure ridge () and pure LASSO (). The combination inherits LASSO's sparsity and ridge's stability under collinearity.
The core mechanism is the grouped selection property. When a block of predictors is pairwise correlated, the elastic net tends to assign them similar coefficient magnitudes and select or drop the entire block together. Pure LASSO cannot do this because the L1 ball's geometry has sharp corners that favor selecting a single representative from each correlated group. Adding the L2 penalty rounds those corners: the elastic net constraint set is a blend of the L1 diamond and the L2 sphere, producing a shape whose corners are smoothed enough to let correlated predictors share weight. Formally, Zou and Hastie proved that the difference in elastic net coefficients between two predictors and is bounded by a function of (their sample correlation), so highly correlated predictors receive nearly equal treatment.
Central bank research groups adopted elastic net quickly. The Bank of England uses it for inflation density forecasting with 100+ candidate predictors (Kapetanios, Marcellino, Papailias, 2021). The Federal Reserve Bank of New York applies elastic net in mixed-frequency nowcasting pipelines where financial and real-activity indicators are collinear. The IMF's World Economic Outlook team uses penalized regressions including elastic net for cross-country growth projections with many correlated institutional and macroeconomic controls. In finance, Gu, Kelly, and Xiu (2020) found elastic net competitive with deeper machine learning methods for predicting stock returns from hundreds of firm characteristics.
The estimator was extended by Friedman, Hastie, and Tibshirani (2010) with the glmnet coordinate descent implementation, which made the full two-dimensional cross-validation grid over computationally tractable. The adaptive elastic net (Zou and Zhang, 2009) adds data-dependent coefficient weights to achieve oracle properties under weaker conditions. The group elastic net penalizes predefined blocks of variables (e.g., all lags of a given indicator), combining group LASSO and group ridge. The sparse group elastic net adds within-group sparsity on top of between-group selection.
Inputs are the standard regression setup: observations, predictors (typically standardized to zero mean and unit variance), and a response vector . In macro applications, ranges from 50 to 500 candidate predictors drawn from financial conditions indices, labor market indicators, survey data, international trade flows, and their lags. The elastic net expects that a moderate-sized subset of these predictors is relevant, possibly in correlated clusters. Unlike LASSO, the analyst does not need to assume that only one predictor per cluster is relevant.
The objective function is . Two hyperparameters govern the fit: the overall penalty strength and the mixing parameter . At , the penalty is pure L1 (LASSO). At , the penalty is pure L2 (ridge). Intermediate values produce models that are sparse (some coefficients exactly zero) but grouping-stable (correlated predictors share selection fate). The L2 component also removes the cap of on the number of selected variables that LASSO imposes: elastic net can select all predictors if .
Estimation proceeds by coordinate descent (Friedman, Hastie, Tibshirani, 2010). For a fixed , the algorithm cycles through predictors and applies a soft-thresholding update that accounts for both penalties. The update for coordinate is , where is the partial residual regression coefficient and is the soft-thresholding operator. The denominator is the ridge contribution: it further shrinks the surviving coefficients beyond what LASSO alone would do. The algorithm is warm-started along a grid of values for each , making the full path computationally cheap. Hyperparameter selection uses two-dimensional -fold cross-validation over a grid of pairs, or a one-dimensional search over at a few fixed values (0.25, 0.50, 0.75, 1.0) followed by selection of the best pair.
The Bank of England's forecasting division uses elastic net to select from 150+ monthly economic indicators when building inflation fan charts. The grouped selection property matters here: inflation expectations from surveys, market breakevens, and model-implied measures are highly correlated, and the policy team needs all members of a relevant group to appear in the model rather than having LASSO arbitrarily pick one. The selected model typically retains 15 to 30 variables in 4 to 6 correlated clusters, which maps cleanly onto the narrative structure the Monetary Policy Committee expects.
In cross-country growth regressions, elastic net resolves the "open-endedness" problem identified by Levine and Renelt (1992): dozens of institutional, geographic, and policy variables are plausible growth determinants, many are correlated, and OLS cannot handle them all simultaneously. Elastic net selects a stable subset while keeping correlated governance indicators (rule of law, corruption control, regulatory quality) together rather than dropping all but one. The IMF and World Bank have published working papers using this approach for policy-relevant variable selection in development economics.
Financial risk management uses elastic net for portfolio construction with hundreds of assets. DeMiguel, Martin-Utrera, Nogales (2018) showed that elastic net portfolios outperform LASSO portfolios out of sample because the L2 penalty stabilizes weights among correlated assets (e.g., stocks in the same sector), reducing turnover and transaction costs.
Elastic net should not be used when predictors are genuinely orthogonal and very sparse selection is needed: the L2 penalty adds unnecessary bias in that setting, and pure LASSO is strictly better. It also should not be used when the practitioner needs valid post-selection confidence intervals, because the selective inference literature for elastic net is less developed than for LASSO. Finally, if the goal is pure prediction without any sparsity requirement, ridge or random forests may dominate because they avoid the bias introduced by L1 thresholding on weakly relevant predictors.
The vector of penalized coefficients. Some entries are exactly zero (sparsity from L1); nonzero entries are shrunk toward zero (from both L1 and L2).
Non-negative scalar controlling the total amount of regularization. Larger produces sparser, more shrunken models.
Scalar in controlling the balance between LASSO () and ridge (). Intermediate values yield grouped selection with sparsity.
. Sets to zero if ; otherwise shrinks toward zero by .
The set of predictors with nonzero elastic net coefficients: . Unlike LASSO, this set can exceed in size when .
Approximated by the number of nonzero coefficients divided by the ridge shrinkage factor: . Interpolates between LASSO df (count of nonzeros) and ridge df (continuous trace).
The true coefficient vector has a moderate number of nonzero entries, possibly clustered among correlated predictors. The truth need not be as sparse as LASSO requires.
If violated: If the truth is extremely sparse with no correlated structure, elastic net's L2 component adds unnecessary bias without improving selection. Pure LASSO is more efficient in that case.
for some coefficient vector .
If violated: Elastic net estimates a linear approximation. Nonlinear effects are absorbed into residuals; important interaction or threshold effects are missed.
All predictors are scaled to zero mean and unit variance before applying the penalty.
If violated: Both L1 and L2 penalties treat coefficients on different scales unequally. A predictor measured in large units is penalized disproportionately. Standardization is required for the grouped selection property to function correctly.
Errors are i.i.d. sub-Gaussian, or weakly dependent with summable mixing coefficients for the time-series case.
If violated: Strong serial correlation invalidates standard -fold cross-validation. Oracle inequalities require adjustment for the reduced effective sample size.
The design matrix satisfies a restricted eigenvalue condition: for all in a cone around the true sparse support.
If violated: Without this condition, the elastic net cannot reliably recover the true support direction. Estimation rates degrade and the grouped selection guarantee weakens.
is bounded away from zero: with for sparsity to hold.
If violated: At the model reduces to ridge with no sparsity. For the elastic net's variable selection properties to hold, must be strictly positive.
Continue reading
Open the concept, data series, policy setting, or neighboring model that anchors this page.