Cookies on Macro by Mark.We use essential cookies for sign-in, locale, and security. With your permission, we also use optional cookies for preferences, for analytics, and for advertising measurement with the Meta (Facebook) Pixel. Read our cookie policy.
Recurrent neural network architecture with gates that can retain or forget information across long macro sequences.
Graph workspace
This page explains what the architecture is, not how one particular fit turned out. Two questions decide whether an LSTM can work on a macro series at all, and both are answered exactly by the design rather than by training: how many parameters the hidden width buys against how few quarterly observations exist to fit them, and how far gradient still reaches once it has been multiplied by a forget gate at every step of the lookback. Running a network and scoring it is a Macro Lab exercise; this graph is for seeing why the answer usually comes out the way it does.
Trainable parameters
Every weight the optimizer has to set. Four gates each carry a recurrent matrix, so the count grows with the SQUARE of the hidden width: 101 parameters at a width of 4, but 66,689 at a width of 128.
4385
count
Effective parameters per observation
Parameters per training example, once dropout has thinned the hidden layer, against the roughly 250 quarters a long macro series offers. Above one there are more free parameters than observations to pin them down. At the default width that ratio is near thirteen, which is the honest reason deep sequence models struggle on macro data.
12.62
ratio
Gradient retention at the deepest lag
The share of gradient still alive after being multiplied by a forget gate at every step of the lookback. With the standard bias-one initialization the gate sits near 0.73, so a 24-step window passes back about of the signal. The gating slows the decay; it does not stop it.
0.0005
proportion
Useful gradient depth
How many steps back still receive an update worth calling nonzero, taking as the floor. Compare it with the lookback: when it falls short, the extra window length is costing parameters without teaching the network anything. A larger learning rate pushes the useful depth outward, which is one of the few honest ways to buy reach.
22.00
steps
Hidden dimension: 32
How much pattern the network can hold; bigger settings need more data to train.
Larger : training loss drops faster and lower, validation loss improves initially but may diverge sooner if overfitting. Forecast tracks actual series more closely up to a point.
Lookback window: 24
How many past periods the model sees each time it makes a forecast.
Longer lookback: the network can capture longer cycles, but training slows and overfitting risk increases on short series. Forecast may improve at turning points where long-range context helps.
Learning rate: 0.001
How much the model adjusts itself at every training step.
Higher learning rate: faster initial loss drop but noisier convergence and earlier overfitting. Lower learning rate: slower, smoother convergence curves requiring more epochs.
Dropout rate: 0.2
The share of the network switched off at random during training, which curbs memorizing the data.
Higher dropout: training loss stays higher (regularization effect), but the gap between training and validation loss shrinks. Forecast becomes smoother and less reactive to noise.
Noise level: 0.5
How much random noise sits in the data, hiding the pattern. It changes the chart only; the readouts describe the network's architecture.
Higher noise makes the plotted target series swing more widely. The parameter and gradient panels do not move: the page trains nothing, so there are no loss curves for noise to raise.