The Accuracy of CPI Inflation Forecasts
In a recent study we compared the accuracy of different techniques for forecasting future CPI inflation and found that forecast accuracy varied a lot over time and technique. Here, we provide more evidence on the variation of forecast accuracy over time.
Our study involved constructing many different forecasts of one-year-ahead annual CPI inflation using a number of variables and methods and then comparing the accuracy of those forecasts. Variables included CPI inflation, core measures of inflation, measures of economic activity, and inflation expectations obtained from surveys. We incorporated these variables into regressions or simply took their most recent values as the forecast (the so-called “naïve” method). To assess forecast performance, we used the “root mean square error (RMSE)” statistic. The RMSE is the average squared forecast error over the forecast sample, so the higher the RMSE, the higher the deviation (squared) between the forecasted values and the realized values on average. The table below summarizes some of the results of this exercise.
|Regression with CPI inflation||0.98||2.18||2.31||0.96||2.39|
|Regression with the annualized growth rate of real GDP||0.93||2.09||2.15||0.88||2.39|
|Regression with median CPI inflation||2.48||1.22||1.57|
|“Naïve” forecast with 16% trimmed-mean CPI inflation |
over the past four quarters
|Regression with SPF one-year inflation expectations||1.37|
|“Naïve” forecast with University of Michigan one-year |
Sources: Bureau of Economic Analysis, Bureau of Labor Statistics, Federal Reserve Bank of Philadelphia, University of Michigan Survey of Consumers and authors' calculations.
As the RMSEs show, the accuracy of inflation forecasts has varied a lot over time. Inflation forecasts were not very accurate in the 1970s and 1980s. Accuracy improved in the 1990s and began to fall again in the 2000s.
The RMSE statistic uses a sample time period (like a decade above) and provides one measure for this sample. However, we want to look at forecast accuracy in finer detail (at a higher frequency) to analyze its variation over time more closely. One way to get a measure of accuracy at a higher frequency is to use what statisticians call the “rolling RMSE” of forecasts. A rolling RMSE statistic allows us to get a measure of forecast accuracy for each point in the forecast sample. As the name suggests, the rolling RMSE statistic uses a sample window consisting of a current point in time and a fixed number of past and future values around that point, and as the window moves forward in time, older observations roll off and more recent observations are added on. In addition, instead of treating all observations within a sample window equally, a rolling RMSE statistic uses a weighting scheme that gives the most weight to the current period and less weight to other observations in the sample (the weight declines the farther the observation is from the current period, with lagged and lead values treated symmetrically). Since the data are quarterly and we use 7 lags and 7 leads, we have rolling RMSE statistics up until the third quarter of 2008.
Relative to the 1960s, the rolling RMSEs of the different forecasts increased considerably during the volatile period of the 1970s. They stayed at high levels in the early 1980s, indicating the relative inaccuracy of inflation forecasts during this time. This period is notable in that a significant and sudden disinflation was occurring, associated with the policies of the Volcker-led Federal Reserve, and the deterioration in forecast accuracy is consistent with the fact that during periods of marked change (like disinflations), methods that are based on past relationships and use past data have a hard time predicting the future. By contrast, methods that are based on asking people about their expectations are typically more successful at such times since people can adjust to policy changes more quickly. For example, naïve forecasts using expectations from the University of Michigan Consumer Survey in the 1981-1986 period dominated other forecasts, though even that method of forecasting took some time to adjust to the changed environment and predict more accurately.
There are two important implications from these analyses for the accuracy of recent inflation forecasts. First, the rapid increase in the rolling RMSEs in 2008 is mainly due to the considerable deviation that emerged between the predicted and realized values of inflation during the period between the summer of 2008 and the end of 2009. This seems to be related to the large shocks in the economy during this period. First, rising energy prices led to higher CPI inflation in 2008, although this was quickly reversed once the financial crisis hit. Note that the rolling RMSEs started using these observations only from 2007 onward and to a higher degree for the later periods. The second takeaway is that the accuracy of forecasts improved considerably in the first two quarters of 2010 as these shocks subsided.