Sepp Hochreiter and Jurgen Schmidhuber introduced the Long Short-Term Memory network in 1997 to solve a specific, well-characterized failure mode: the vanishing gradient problem in recurrent neural networks. Bengio, Simard, and Frasconi (1994) had already shown formally that gradients in vanilla RNNs shrink exponentially with sequence length during backpropagation through time (BPTT), making it impossible for the network to learn dependencies spanning more than roughly 10 to 20 time steps. Hochreiter's 1991 diploma thesis diagnosed the root cause as the repeated multiplication of the recurrent weight matrix, whose spectral radius determines whether gradients vanish (spectral radius < 1) or explode (spectral radius > 1). The LSTM solved this by introducing a dedicated cell state $c_t$ that flows through time with additive rather than multiplicative updates, and three multiplicative gates that regulate what information enters, persists in, and exits the cell.

The core mechanism is the cell state highway. At each time step $t$, the cell state $c_{t-1}$ from the previous step passes through a forget gate $f_t$ that element-wise scales it, then receives new candidate information $\tilde{c}_t$ scaled by an input gate $i_t$: $c_t = f_t \odot c_{t-1} + i_t \odot \tilde{c}_t$. The forget gate is a sigmoid-activated linear combination of the current input $x_t$ and previous hidden state $h_{t-1}$: values near 1 preserve the cell content, values near 0 erase it. The input gate similarly controls how much of the new candidate (produced by a tanh-activated transform of $x_t$ and $h_{t-1}$) is written into the cell. An output gate $o_t$ then selects which parts of the (tanh-squashed) cell state are exposed as the hidden state $h_t = o_t \odot \tanh(c_t)$. This gating mechanism means the gradient can flow through the cell state with near-unity magnitude across many time steps, solving the vanishing gradient problem by construction.

LSTMs became the dominant sequence model in deep learning from roughly 2014 to 2018, spanning machine translation (Sutskever, Shvartsman, and Hinton, 2014), speech recognition (Graves, Mohamed, and Hinton, 2013), and time-series forecasting. In macroeconomics, LSTMs have been deployed for GDP nowcasting, inflation forecasting, exchange rate prediction, and yield curve modeling. Nakamura (2005) was among the early applications to economic time series. More recently, the Federal Reserve Bank of New York and the Bank of Canada have experimented with LSTM-based nowcasting models that process high-frequency mixed-frequency inputs (daily financial indicators combined with monthly macro releases) in a sequential framework that naturally handles irregular arrival times. Makridakis, Spiliotis, and Assimakopoulos (2018) tested LSTMs in the M4 forecasting competition and found them competitive with statistical methods on longer series but less reliable on short ones, a pattern consistent with the data-hungry nature of neural sequence models.

The arrival of the Transformer architecture (Vaswani et al., 2017) reduced LSTM's dominance in NLP, but LSTMs remain competitive for univariate and low-dimensional multivariate time-series forecasting where the sequence is genuinely temporal and attention over thousands of positions is unnecessary. Salinas, Flunkert, Gasthaus, and Januschowski (2020) showed that LSTM-based DeepAR models produce well-calibrated probabilistic forecasts across thousands of related time series in Amazon's demand forecasting pipeline. For macro applications with moderate sequence lengths (60 to 240 monthly observations), LSTMs often outperform Transformers because the inductive bias of sequential processing matches the autoregressive structure of economic data, and the parameter count is lower, reducing overfitting risk on small samples.

Inputs to an LSTM for macro forecasting are typically multivariate time-series windows: at each time step $t$, the input vector $x_t \in \mathbb{R}^p$ contains $p$ features (GDP growth, inflation, unemployment rate, interest rates, financial condition indices, and their transformations). The practitioner selects a lookback window of $T$ steps, producing an input tensor of shape $(T, p)$ for each forecast. Nonstationary series are typically differenced or standardized before feeding into the network. Missing values require imputation or masking, since the LSTM processes every step sequentially and cannot skip positions the way tree-based methods handle missing splits. For mixed-frequency applications, higher-frequency inputs (daily or weekly) are aggregated or the LSTM is paired with an encoder that processes each frequency stream separately before merging at a lower-frequency bottleneck.

The LSTM cell processes inputs sequentially from $t = 1$ to $t = T$. At each step, four linear transformations are computed in parallel: $W_f [h_{t-1}, x_t] + b_f$ for the forget gate, $W_i [h_{t-1}, x_t] + b_i$ for the input gate, $W_c [h_{t-1}, x_t] + b_c$ for the candidate cell state, and $W_o [h_{t-1}, x_t] + b_o$ for the output gate. The forget and input gates pass through sigmoid activations ($\sigma$), the candidate through tanh, and the output gate through sigmoid. The cell state update is $c_t = f_t \odot c_{t-1} + i_t \odot \tilde{c}_t$, and the hidden state is $h_t = o_t \odot \tanh(c_t)$. The final hidden state $h_T$ (or a pooled representation of all hidden states) is fed through one or more dense layers to produce the forecast $\hat{y}_{T+h}$ at horizon $h$. Stacking multiple LSTM layers (2 to 3 is typical) allows the network to learn hierarchical temporal representations, where lower layers capture short-range patterns and upper layers capture longer-range dynamics.

Training uses backpropagation through time (BPTT), which unrolls the recurrent computation graph across $T$ time steps and applies standard backpropagation to compute gradients of the loss with respect to all weight matrices and biases. The loss function is typically MSE for point forecasts or negative log-likelihood for probabilistic forecasts (Gaussian, Student-t, or mixture outputs). Optimization is performed via Adam (Kingma and Ba, 2015) or a variant with gradient clipping (typically clipping the global gradient norm to 1.0 or 5.0) to prevent the exploding gradient problem. Regularization includes dropout applied to the non-recurrent connections (Gal and Ghahramani, 2016 introduced variational dropout that applies the same mask at every time step), weight decay (L2 penalty on all parameters), and early stopping on a temporal validation set. The learning rate schedule often includes warm-up followed by cosine annealing or reduce-on-plateau.

The Federal Reserve Bank of Atlanta has explored LSTM architectures for GDPNow-style nowcasting, where high-frequency data releases arrive sequentially throughout the quarter and the model must update its estimate after each new input. The LSTM's sequential processing is a natural fit: each new data release is a new time step, and the hidden state accumulates evidence from all prior releases. Compared to the bridge equation approach that re-estimates a static regression at each release, the LSTM can learn nonlinear interactions between the ordering and timing of releases. Banbura, Giannone, Modugno, and Reichlin (2013) established the nowcasting baseline with dynamic factor models; LSTM extensions aim to capture nonlinear dynamics that factor models miss, particularly around turning points where the relationship between high-frequency indicators and GDP growth changes character.

In financial macro applications, LSTMs have been applied to yield curve forecasting, exchange rate prediction, and volatility modeling. Heaton, Polson, and Witte (2017) demonstrated LSTM-based portfolio construction using sequences of asset returns and macro indicators. The key advantage over static models is the ability to condition on the recent trajectory of indicators as well as their current levels. For example, an LSTM can learn that rapidly rising unemployment combined with falling consumer confidence predicts recession onset differently than the same levels reached gradually. This path-dependence is difficult to encode in traditional econometric models without hand-crafting interaction terms and lag structures. The Bank of England's machine learning forecasting toolkit includes LSTM models alongside gradient-boosted trees and Bayesian VARs, with the LSTM performing best on horizons of 1 to 4 quarters where sequential momentum effects dominate.

LSTMs should not be used when the available time series is very short (fewer than 100 observations for univariate, fewer than 200 for multivariate), because the parameter count relative to sample size makes overfitting nearly inevitable even with heavy regularization. They should also be avoided when interpretability is a hard requirement: LSTM hidden states are dense vectors with no direct economic interpretation, and post-hoc attribution methods (integrated gradients, attention-based explanations) provide only approximate, sometimes misleading, feature importance rankings. For applications requiring structural coefficients (fiscal multipliers, Phillips curve slopes, monetary policy rule parameters), VAR or SVAR models with economic identification restrictions are appropriate. When the temporal structure is simple (low-order autoregressive dynamics with no nonlinear interactions), ARIMA or ETS models will match LSTM accuracy at a fraction of the complexity. See the gradient boosting reference at /models/empirical/gradient-boosting for the primary non-sequential ML alternative that handles tabular macro data with less architectural overhead.

Probabilistic extensions of LSTMs produce full predictive distributions rather than point forecasts. Salinas et al. (2020) introduced DeepAR, which uses an LSTM encoder to produce the parameters of a parametric output distribution (Gaussian, negative binomial, or Student-t) at each forecast step. This probabilistic framework enables density forecasting, quantile estimation, and calibrated prediction intervals. For central bank applications where the distribution of GDP growth or inflation matters as much as the point estimate (e.g., fan charts in Monetary Policy Reports), probabilistic LSTMs provide a data-driven alternative to the ad hoc distributional assumptions layered on top of traditional point-forecast models.