Cookies on Macro by Mark.We use essential cookies for sign-in, locale, and security. With your permission, we also use optional cookies for preferences, for analytics, and for advertising measurement with the Meta (Facebook) Pixel. Read our cookie policy.
Sequential tree ensemble that fits residual structure stage by stage and uses validation loss to control overfitting.
Graph workspace
The training-vs-validation loss plot is the canonical gradient boosting diagnostic. It directly visualizes the overfitting trajectory: training loss monotonically decreases while validation loss decreases, reaches a minimum, and then rises. The gap between the two curves and the location of the optimal stopping round answer the two key practitioner questions: how complex should the model be, and how much overfitting risk remains.
Optimal stopping round
The boosting round where validation loss is minimized. At a given learning rate an earlier stop means noise is fitted sooner; a smaller learning rate needs more rounds for the same signal, so a late stop alone is not a better fit (the conservative preset stops latest, at round 389, with a validation loss of about 1.27, above the baseline's 1.21 at round 13). With only 48 validation rows the round itself is noisy.
13.00
rounds
Validation MSE at early stop
The best achievable validation error. Compare against the noise floor to gauge how much signal the model captures.
1.21
squared units
Train-validation gap at early stop
Training loss minus validation loss, both at the early-stopping round. It is negative whenever the model fits the data it trained on better than held-out data, which boosting almost always does; the more negative it is, the more training noise the model has memorized by the time validation loss bottoms out.
-0.1966
squared units
Overshoot ratio
Ratio of validation loss at the final round to the optimum. Values above 1.0 quantify how much accuracy is lost by failing to stop early.
1.06
ratio
The most rounds allowed; training stops early anyway, so this mainly sets how far the curves run.
Extending M shows more of the overfitting trajectory. The validation curve's post-minimum rise becomes visible with enough rounds.
How much each added step counts; smaller steps need more rounds but behave more smoothly on new data.
Low learning rate: both curves descend slowly and the gap between them stays small. High learning rate: training loss drops fast, validation loss diverges earlier.
Leaves per tree: 4
How many end-points each tree has; bigger trees can combine more variables and are stronger but noisier.
Trees with more leaves cause training loss to drop faster and validation loss to climb further after its minimum (an overshoot ratio of about 1.16 at 8 leaves against 1.06 at 4). Smaller trees converge more slowly. On this small validation set the stopping round itself does not move steadily with tree size.
Subsample fraction: 0.8
The share of training rows used each round; lower shares add random variation that restrains the fit.
Lower subsampling adds randomness that can delay the onset of overfitting, but with only 48 validation rows the round of the validation minimum jumps around as changes (from 11 to about 360 rounds across the subsample slider at the defaults, 357 at 0.4) rather than moving steadily. Training loss may be slightly noisier.
How much random noise the data contains; more noise raises the error no model can remove.
Higher noise raises both curves. The validation curve minimum tends to occur at fewer rounds because the model exhausts learnable signal sooner (about 370 to 390 rounds at to 0.6, 11 or fewer from up).