Skip to main content

Cookies on Macro by Mark.We use essential cookies for sign-in, locale, and security. With your permission, we also use optional cookies for preferences, for analytics, and for advertising measurement with the Meta (Facebook) Pixel. Read our cookie policy.

Macro by MarkMacro by Mark
  • Home
Loading…
Loading…
← Models
MacroConceptsPolicyModels

Ridge regression Model

L2-penalized linear regression - shrinks coefficients toward zero to stabilize forecasts when predictors are many or collinear.

OverviewGraphMathReferences

Graph workspace

Coefficient shrinkage path as a function of the penalty parameter

Graph workspace

The ridge trace - coefficient values plotted against λ\lambdaλ (or equivalently, effective degrees of freedom) - is the canonical visualization. It shows how each coefficient shrinks toward zero as regularization increases, and reveals which predictors are stable (coefficients barely move) versus unstable (coefficients swing wildly at low λ\lambdaλ). No other plot captures the bias-variance tradeoff as directly.

Chart axis metadata
xPenalty step: 0 is the strongest penalty, each step to the right relaxes it, 11 is a thousandth of itstep on a 12-point log grid of the penalty · 0 to 11yCoefficient valuestandardized units · -3.0 to 3.0
Coefficient paths as the ridge penalty
is relaxed (strongest at the left) static chart preview showing Coefficient 1, Coefficient 2, Coefficient 3, Coefficient 4, Coefficient 5
Eigenvalues of the design: one large
root under collinearity static chart preview showing Series 1
xPenalty step: 0 is the strongest penalty, each step to the right relaxes it, 11 is a thousandth of itstep on a 12-point log grid of the penalty · 0 to 11yCoefficient valuestandardized units · -3.0 to 3.0

Current results

OLS MSE

Explain: OLS MSE
p−1∥β^OLS−βtrue∥2p^{-1}\|\hat{\beta}_{\text{OLS}} - \beta_{\text{true}}\|^2p−1∥β^​OLS​−βtrue​∥2

Mean squared error of the unpenalized OLS coefficients against the true ones, averaged over the p coefficients. Baseline against which ridge improvement is measured.

0.0605

squared standardized units

Ridge MSE at optimal lambda

Explain: Ridge MSE at optimal lambda
p−1∥β^ridge(λ∗)−βtrue∥2,λ∗=arg⁡min⁡λ∈{0}∪Λp−1∥β^ridge(λ)−βtrue∥2p^{-1}\|\hat{\beta}_{\text{ridge}}(\lambda^*) - \beta_{\text{true}}\|^2, \quad \lambda^* = \arg\min_{\lambda \in \{0\} \cup \Lambda} p^{-1}\|\hat{\beta}_{\text{ridge}}(\lambda) - \beta_{\text{true}}\|^2p−1∥β^​ridge​(λ∗)−βtrue​∥2,λ∗=argminλ∈{0}∪Λ​p−1∥β^​ridge​(λ)−βtrue​∥2

Mean squared coefficient error of ridge at the oracle penalty: of zero (OLS itself) and the 12-point penalty grid Λ\LambdaΛ, the penalty whose coefficients come closest to the true ones. Only a synthetic sample, where the true coefficients are known, can choose λ\lambdaλ this way; with real data GCV or cross-validation picks it and does somewhat worse. Never above the OLS MSE, and equal to it when no penalty brings this sample's coefficients closer to the truth, as at the defaults; ridge's expected gain under collinearity is an average over samples, and on this one it shows from ρ≈0.9\rho \approx 0.9ρ≈0.9 up.

0.0605

squared standardized units

Effective degrees of freedom

Explain: Effective degrees of freedom
df(λ∗)=∑jdj2/(dj2+λ∗)\text{df}(\lambda^*) = \sum_j d_j^2 / (d_j^2 + \lambda^*)df(λ∗)=∑j​dj2​/(dj2​+λ∗)

Number of effective parameters at the selected penalty, computed from the design's population eigenvalues (one root 1+(p−1)ρ1 + (p-1)\rho1+(p−1)ρ, the other p−1p - 1p−1 at 1−ρ1 - \rho1−ρ). Equal to ppp when the oracle penalty is zero (as at the defaults), and lower by however much complexity the penalty removes.

6.00

dimensionless

Shrinkage ratio

Explain: Shrinkage ratio
1−∥β^ridge(λ∗)∥2/∥β^OLS∥21 - \|\hat{\beta}_{\text{ridge}}(\lambda^*)\|_2 / \|\hat{\beta}_{\text{OLS}}\|_21−∥β^​ridge​(λ∗)∥2​/∥β^​OLS​∥2​

Share of the OLS coefficient length the penalty removes: 0 means no shrinkage, values near 1 mean the coefficients are pulled almost to zero. At 0.05, ridge kept 95 percent of the OLS length; it is exactly 0 when the oracle penalty is zero.

0.0000

proportion

Controls

predictors overlap
Explain: How alike the predictors are

Predictor correlation: 0.8

How closely the predictors move together; the closer they are, the shakier plain regression gets.

High ρ\rhoρ makes the OLS (right) end of the trace erratic - coefficients overshoot, and the benefit of shrinkage becomes more dramatic. On this sample the oracle penalty is zero for every ρ\rhoρ from 0.03 to 0.88, the default 0.8 included, and positive from 0.89 up; at 0.99 ridge cuts the coefficient error by more than four fifths.

1
Explain: How noisy the data is

How much random error clouds the data; more noise makes coefficients harder to pin down.

Increasing σ\sigmaσ moves the oracle penalty toward the stronger penalties at the left of the path panel (more shrinkage pays): at the default correlation it stays at zero up to about σ=1.2\sigma = 1.2σ=1.2 and rises after.

6
Explain: How many predictors

How many predictor variables the regression uses; more of them add estimation uncertainty.

The panel draws the first five coefficient paths whatever p is. On this one fixed sample the oracle penalty does not move steadily with p, because each p draws a different design.

80
Explain: How many observations

How many data points there are; with more, coefficients need less pulling toward zero.

Increasing n makes the OLS end more stable. Less shrinkage is needed on average, but on this one fixed sample the oracle penalty jumps around as n changes, because each n draws different noise.

1.5
Explain: Size of the biggest effect

How large the strongest true effect is; all the true effects scale with it.

Larger signals push the coefficient paths further from zero at all λ\lambdaλ values, and the oracle penalty favors less shrinkage (at the default correlation it is zero from βmax⁡=1.25\beta_{\max} = 1.25βmax​=1.25 up).

Keep going

Read next

Continue with a related concept, model, or example.

Theory model

IS-LM Model

a well-connected theoretical model two steps from here

New angle: theoretical model →
Macro by Mark
Help
Privacy PolicyTerms of UseEthics and Compliance

© 2026 Mark Jayson Nation LLC