Skip to main content

Cookies on Macro by Mark.We use essential cookies for sign-in, locale, and security. With your permission, we also use optional cookies for preferences, for analytics, and for advertising measurement with the Meta (Facebook) Pixel. Read our cookie policy.

Macro by MarkMacro by Mark
  • Home
Loading…
Loading…
← Models
MacroConceptsPolicyModels

Gradient boosting Model

Sequential tree ensemble that fits residual structure stage by stage and uses validation loss to control overfitting.

OverviewGraphMathReferences

Graph workspace

Training vs. validation loss curves with early stopping

Graph workspace

The training-vs-validation loss plot is the canonical gradient boosting diagnostic. It directly visualizes the overfitting trajectory: training loss monotonically decreases while validation loss decreases, reaches a minimum, and then rises. The gap between the two curves and the location of the optimal stopping round answer the two key practitioner questions: how complex should the model be, and how much overfitting risk remains.

Chart axis metadata
xBoosting roundcount · 1 to 1000yLoss (MSE)squared units of y · Depends on response scale; typical macro: 0.2 to 5.0
Training and validation error across
boosting rounds static chart preview showing Training error, Validation error
xBoosting roundcount · 1 to 1000yLoss (MSE)squared units of y · Depends on response scale; typical macro: 0.2 to 5.0

Current results

Optimal stopping round

Explain: Optimal stopping round
m∗=arg⁡min⁡mLval(m)m^* = \arg\min_m L_{\text{val}}(m)m∗=argminm​Lval​(m)

The boosting round where validation loss is minimized. At a given learning rate an earlier stop means noise is fitted sooner; a smaller learning rate needs more rounds for the same signal, so a late stop alone is not a better fit (the conservative preset stops latest, at round 389, with a validation loss of about 1.27, above the baseline's 1.21 at round 13). With only 48 validation rows the round itself is noisy.

13.00

rounds

Validation MSE at early stop

Explain: Validation MSE at early stop
Lval(m∗)L_{\text{val}}(m^*)Lval​(m∗)

The best achievable validation error. Compare against the noise floor σ2\sigma^2σ2 to gauge how much signal the model captures.

1.21

squared units

Train-validation gap at early stop

Explain: Train-validation gap at early stop
Ltrain(m∗)−Lval(m∗)L_{\text{train}}(m^*) - L_{\text{val}}(m^*)Ltrain​(m∗)−Lval​(m∗)

Training loss minus validation loss, both at the early-stopping round. It is negative whenever the model fits the data it trained on better than held-out data, which boosting almost always does; the more negative it is, the more training noise the model has memorized by the time validation loss bottoms out.

-0.1966

squared units

Overshoot ratio

Explain: Overshoot ratio
Lval(M)  /  Lval(m∗)L_{\text{val}}(M) \;/\; L_{\text{val}}(m^*)Lval​(M)/Lval​(m∗)

Ratio of validation loss at the final round to the optimum. Values above 1.0 quantify how much accuracy is lost by failing to stop early.

1.06

ratio

Controls

400
Explain: How many rounds to run

The most rounds allowed; training stops early anyway, so this mainly sets how far the curves run.

Extending M shows more of the overfitting trajectory. The validation curve's post-minimum rise becomes visible with enough rounds.

0.1
Explain: Size of each learning step

How much each added step counts; smaller steps need more rounds but behave more smoothly on new data.

Low learning rate: both curves descend slowly and the gap between them stays small. High learning rate: training loss drops fast, validation loss diverges earlier.

trees combine a few variables
Explain: How big each tree is

Leaves per tree: 4

How many end-points each tree has; bigger trees can combine more variables and are stronger but noisier.

Trees with more leaves cause training loss to drop faster and validation loss to climb further after its minimum (an overshoot ratio of about 1.16 at 8 leaves against 1.06 at 4). Smaller trees converge more slowly. On this small validation set the stopping round itself does not move steadily with tree size.

rounds use most of the data
Explain: Share of data per round

Subsample fraction: 0.8

The share of training rows used each round; lower shares add random variation that restrains the fit.

Lower subsampling adds randomness that can delay the onset of overfitting, but with only 48 validation rows the round of the validation minimum jumps around as η\etaη changes (from 11 to about 360 rounds across the subsample slider at the defaults, 357 at 0.4) rather than moving steadily. Training loss may be slightly noisier.

1
Explain: How noisy the data is

How much random noise the data contains; more noise raises the error no model can remove.

Higher noise raises both curves. The validation curve minimum tends to occur at fewer rounds because the model exhausts learnable signal sooner (about 370 to 390 rounds at σ=0.2\sigma = 0.2σ=0.2 to 0.6, 11 or fewer from σ=2\sigma = 2σ=2 up).

Keep going

Read next

Continue with a related concept, model, or example.

Theory model

IS-LM Model

a well-connected theoretical model two steps from here

New angle: theoretical model →
Macro by Mark
Help
Privacy PolicyTerms of UseEthics and Compliance

© 2026 Mark Jayson Nation LLC