Cookies on Macro by Mark.We use essential cookies for sign-in, locale, and security. With your permission, we also use optional cookies for preferences, for analytics, and for advertising measurement with the Meta (Facebook) Pixel. Read our cookie policy.
Regularized regression that blends LASSO selection with ridge shrinkage for correlated predictor blocks.
Follow the same equations and assumptions one section at a time or in a continuous reading.
Hover or focus a symbol defined in this model to find its exact matches. Select it to keep the highlight; Escape clears the selection.
The elastic net objective augments the OLS loss with a convex combination of L1 and L2 penalties. The resulting constraint set is the intersection of an L1 ball and an L2 ball, producing a shape with rounded corners that interpolates between the LASSO diamond and the ridge sphere.
Zou and Hastie (2005) noted that the naive elastic net (solving the penalized problem directly) suffers from double shrinkage: the L2 penalty shrinks coefficients, and then the L1 penalty shrinks them again. They proposed a rescaling correction: multiply the naive elastic net coefficients by to undo the extra L2 shrinkage. Modern implementations (glmnet) omit this rescaling and solve the naive elastic net directly - the coordinate descent update below is the naive one.
The elastic net objective. The parameter controls the L1/L2 balance, and controls overall penalty strength. At this is LASSO; at this is ridge.
Two-parameter form. Some references use instead of . The reparameterization is one-to-one for .
Lagrangian duality maps the penalty into a constraint. The feasible set is a blend of the L1 diamond and the L2 sphere. Its corners (where sparsity occurs) are smoothed by the L2 component.
Zou-Hastie rescaling. Corrects the double shrinkage of the naive elastic net. Without this, the L2 penalty over-shrinks coefficients that already survived L1 thresholding.
Constraint set geometry in 2D
In two dimensions, the LASSO constraint is a diamond (rotated square), the ridge constraint is a circle, and the elastic net constraint is a rounded diamond whose curvature at the corners depends on . At , the corners have noticeable rounding. This rounding is what prevents the OLS ellipse from landing exactly at a corner where one coordinate is zero while the other is large. Instead, the ellipse touches a smoothed face where both coordinates can be nonzero.
Augmented data trick
The augmented data construction is operational, not a proof device. It means any software that solves LASSO can solve elastic net by preprocessing the data. This is how early implementations worked before dedicated coordinate descent code existed. The augmentation adds p pseudo-observations, so the effective sample size is n + p, which is why elastic net can select more than n variables (the augmented problem has n + p rows).
Conditioning on information at T separates the sample from the projection. The forecast can be smooth even when later realized data will not be.
The bridge point is shared so the eye can see the handoff. After that point, the red line is a model forecast, not observed history.
Do not score the forecast segment as realized data or read it as a policy promise.
Each future innovation contributes through its impulse weight . Unless the future is degenerate, longer horizons carry more conditional variance.
The band widens when shocks accumulate through the model recursion.
Do not treat the band edge as an alternative deterministic scenario.
The history segment is realized data available at the forecast origin.
The forecast path is a conditional expectation, so it can be smooth while future realizations remain volatile.
Band width is horizon-specific uncertainty, not a second deterministic transition.
Proof lineage
Typed relationships across modelsParents -- models this proof draws from