Cookies on Macro by Mark.We use essential cookies for sign-in, locale, and security. With your permission, we also use optional cookies for preferences, for analytics, and for advertising measurement with the Meta (Facebook) Pixel. Read our cookie policy.
L1-penalized linear regression - drives some coefficients to zero, doing variable selection inside the fit.
Follow the same equations and assumptions one section at a time or in a continuous reading.
Hover or focus a symbol defined in this model to find its exact matches. Select it to keep the highlight; Escape clears the selection.
The LASSO objective combines the OLS residual sum of squares with an L1 penalty on the coefficient vector. The L1 penalty is not differentiable at zero, which is precisely what enables exact zero-setting of coefficients.
The subgradient optimality condition yields the soft-thresholding rule for each coordinate. This is the foundation of coordinate descent, the workhorse algorithm for computing the LASSO path.
The LASSO objective. The normalization is conventional and ensures the gradient of the squared-error term is .
Subgradient optimality condition for coordinate . The subdifferential of is when and when .
Coordinate-wise update. With standardized predictors (), this simplifies to where is the univariate regression coefficient of the partial residual on .
L1 geometry
In two dimensions, the L1 constraint set is a diamond (rotated square). The OLS contours (ellipses) are most likely to first touch this diamond at a corner, where one coordinate is zero. In dimensions, the L1 ball has corners and faces - the higher the dimension, the more likely the solution lands at a vertex or low-dimensional face, setting many coordinates to zero.
Conditioning on information at T separates the sample from the projection. The forecast can be smooth even when later realized data will not be.
The bridge point is shared so the eye can see the handoff. After that point, the red line is a model forecast, not observed history.
Do not score the forecast segment as realized data or read it as a policy promise.
Each future innovation contributes through its impulse weight . Unless the future is degenerate, longer horizons carry more conditional variance.
The band widens when shocks accumulate through the model recursion.
Do not treat the band edge as an alternative deterministic scenario.
The history segment is realized data available at the forecast origin.
The forecast path is a conditional expectation, so it can be smooth while future realizations remain volatile.
Band width is horizon-specific uncertainty, not a second deterministic transition.