Data-Driven Models
Data-Driven Models
Empirical forecasting models · Model guide
Trend-cycle decomposition for a single macro series, comparing smooth HP trend extraction with Beveridge-Nelson permanent-transitory de...
How do you separate the trend from the cycle in a macroeconomic time series when you have no structural model of the data-generating process?
Robert Hodrick and Edward Prescott circulated their filter as a Carnegie Mellon working paper in 1980, though the published version appeared in the Journal of Money, Credit and Banking only in 1997. The idea is simple: decompose a time series into a trend tau_t and a cyclical residual tau_t by finding the trend that minimizes a weighted sum of fit (how close tau_t stays to and smoothness (how small the second differences of tau_t are). The single tuning parameter lambda controls the trade-off: lambda = 0 reproduces the original series, lambda -> infinity yields a linear time trend. Kydland and Prescott (1990) popularized lambda = 1600 for quarterly data in the real business cycle literature, and the filter became the default detrending tool across central banks, international organizations, and academic macro.
Beveridge and Nelson (1981) took a fundamentally different approach. Rather than imposing an external smoother, they showed that any I(1) process with a stationary first difference can be decomposed into a random walk with drift (the permanent component) plus a stationary residual (the transitory component). The permanent component equals the series minus the sum of all expected future changes: the level the series would reach if it immediately converged to its long-run forecast path. No tuning parameter is needed. The decomposition follows directly from the Wold representation of the first-differenced series, which in practice means fitting an ARIMA model and computing cumulative impulse responses.
The two methods produce strikingly different decompositions in practice. The HP filter yields a smooth trend and a sizable, persistent cycle, which aligns with the RBC tradition's view of large transitory business-cycle fluctuations. The BN decomposition typically yields a volatile trend and a small, quickly-reverting cycle, because the permanent component absorbs most of the series' variation. This tension was clarified by Morley, Nelson, and Zivot (2003), who showed that the standard UCM with uncorrelated trend-cycle shocks maps to the HP filter's smooth-trend result, while the unrestricted UCM with correlated shocks maps to the BN decomposition's volatile-trend result. The difference is not about method but about whether trend and cycle innovations are allowed to covary.
Hamilton (2018) mounted a forceful critique of the HP filter, documenting its tendency to create spurious cyclical dynamics in integrated or near-integrated data. He proposed a regression-based alternative: regress on and take the residual as the cycle. The Hamilton filter avoids end-point bias by construction (it uses only past data), but it targets a specific forecast horizon h rather than extracting a smooth trend. The debate between HP advocates, BN practitioners, and Hamilton-filter proponents remains active. Each method answers a subtly different question about what 'trend' means.
The HP filter requires a single observed time series of length T and a smoothing parameter lambda. The filter solves a penalized least-squares problem: minimize over {tau_t} the sum of squared deviations plus lambda times the sum of squared second differences of tau_t. The solution is a linear filter: tau = , where K is the (T-2) x T second-difference matrix. The cyclical component is tau_t. For quarterly data, lambda = 1600 is standard; for annual data, lambda = 6.25 (Ravn and Uhlig 2002); for monthly data, lambda = 129,600. These values produce approximately equivalent smoothness when adjusted for the data frequency.
The BN decomposition starts from an ARIMA(p,1,q) representation of . Write Delta mu + psi(L) epsilon_t, where psi(L) is the Wold polynomial and epsilon_t is the innovation. The permanent component is tau_t = tau_{t-1} + mu + psi(1) epsilon_t, where psi(1) = sum of all Wold coefficients is the long-run multiplier. The transitory component is tau_t = -sum_{j=1}^{infinity} psi_j^* epsilon_t, where psi_j^* = sum_{k=j+1}^{infinity} psi_k. In practice, the Wold coefficients come from the estimated ARIMA model. The ARIMA order (p, q) must be selected, typically by AIC or BIC on the first-differenced series.
Both methods share a common structure: they split into tau_t (trend/permanent) and (cycle/transitory) such that tau_t + . The HP filter determines the split by a smoothness penalty on the trend. The BN decomposition determines the split by the long-run forecast of the series. King and Rebelo (1993) showed that the HP filter can be interpreted as the optimal Wiener-Kolmogorov filter for a signal-extraction problem where the trend is a second-order random walk and the cycle is white noise, with the signal-to-noise ratio equal to 1/lambda. This Wiener-filter interpretation connects the HP filter to the model-based BN approach: both are doing signal extraction, but under different assumptions about the spectral shape of the components.
The HP filter is the default detrending tool at the IMF, OECD, European Commission, and most central banks for constructing output gap estimates. The IMF's World Economic Outlook uses HP-filtered GDP to compute the output gap for fiscal surveillance in Article IV consultations. The Congressional Budget Office and the European Commission embed the HP trend in their medium-term fiscal frameworks. Despite Hamilton's critique, the filter persists in institutional practice because it is transparent, reproducible, and computationally trivial.
The BN decomposition sees heaviest use in academic macroeconomics and monetary policy research. Cochrane (1988) used it to measure the persistence of GDP shocks, finding that permanent shocks account for a large fraction of GDP variation. Kamber, Morley, and Wong (2018) extended the BN decomposition to a multivariate setting with auxiliary variables (unemployment, inflation) to sharpen the output gap estimate, producing the 'BN filter' that combines the BN logic with cross-variable information. Central bank research departments at the Reserve Bank of New Zealand and the Reserve Bank of Australia have adopted variants of this approach.
Both methods break down in specific ways. The HP filter produces spurious cycles in integrated or near-integrated data (Hamilton 2018), exaggerates cycle amplitude at sample endpoints (Cogley and Nason 1995), and has no model-based standard errors. The BN decomposition yields implausibly volatile trends when the MA root is close to the unit circle, and can produce near-zero transitory components for series that look obviously cyclical. Neither method handles missing data, mixed frequencies, or seasonal patterns. When these limitations bind, practitioners switch to the UCM framework (Harvey 1989) or band-pass filters (Baxter and King 1999).
Comparative studies by Kaiser and Maravall (2001) and Harvey and Jaeger (1993) documented the HP filter's close relationship to the Wiener-Kolmogorov optimal filter implied by specific UCM structures. This connection means the HP filter is 'model-based' in a narrow sense: it is the optimal linear filter for a particular (highly restrictive) signal-extraction problem. The BN decomposition, conversely, is the optimal decomposition under the assumption that trend and cycle shocks are perfectly correlated. Understanding these polar cases clarifies why the methods disagree and helps practitioners decide which assumptions fit their data.
The smoothed trend from the HP filter. Solves the penalized least-squares problem balancing fit to against smoothness measured by second differences.
The residual tau_t. Captures all variation not attributed to the trend. Typically persistent and sizable for standard lambda values.
Penalty weight on trend roughness. Higher lambda produces a smoother trend and a more volatile cycle. lambda = 1600 is the quarterly-data convention from Hodrick and Prescott (1997).
The random walk with drift implied by the Wold representation: tau_t = tau_{t-1} + mu + psi(1) epsilon_t. Equals the long-run forecast of from the ARIMA model.
The gap between and the BN permanent component: tau_t. Converges to zero as the forecast horizon extends, by construction.
Sum of all Wold coefficients. Determines the permanent effect of a unit innovation on the level of . When psi(1) is large, the permanent component absorbs most variation.
The (T-2) x T matrix such that K tau extracts the second differences of the trend. K'K is the roughness penalty in the HP objective function.
The observed series is the sum of trend (or permanent) and cycle (or transitory) components: tau_t + . No multiplicative or nonlinear interaction.
If violated: Series with level-dependent volatility (e.g., nominal GDP) require log-transformation before applying either method. Without it, the decomposition mixes level and variance effects.
The smoothing parameter lambda is set correctly for the data frequency and the target cycle duration. Standard values: 1600 (quarterly), 6.25 (annual), 129600 (monthly).
If violated: Wrong lambda produces a trend that is either too smooth (absorbing genuine structural change) or too rough (absorbing cyclical variation into the trend). There is no data-driven way to choose lambda within the HP framework itself.
The series is I(1): it has a unit root in levels and stationary first differences. The BN decomposition is defined only for I(1) processes.
If violated: If the series is I(0) (trend-stationary), the BN decomposition attributes all variation to the permanent component and the transitory component is degenerate. If I(2), the method must be applied to first differences, changing the interpretation.
The ARIMA(p,1,q) model for is correctly specified. The Wold coefficients psi_j are consistently estimated from the fitted ARIMA.
If violated: Misspecified ARIMA order biases the long-run multiplier psi(1) and distorts the permanent-transitory split. Over-differencing or under-fitting the MA component are the most common errors.
The HP filter uses both past and future observations at every interior point. At the endpoints, the filter becomes one-sided.
If violated: End-point instability: the HP trend at the most recent observations is unreliable and subject to large revisions as new data arrive. This makes the HP cycle unsuitable for real-time policy analysis without explicit adjustment.
The innovation epsilon_t is Gaussian white noise. The BN decomposition does not require Gaussianity for consistency, but confidence intervals and likelihood-based ARIMA selection assume it.
If violated: Non-Gaussian innovations do not bias the BN point estimates but invalidate standard errors and information criteria for ARIMA order selection. Robust bootstrap intervals are an alternative.
Continue reading
Open the concept, data series, policy setting, or neighboring model that anchors this page.