Regional Economic Sentiment: Constructing Quantitative Estimates from the Beige Book and Testing Their Ability to Forecast Recessions

where $n_t^{Negative}$ represents the total number of negative sentences in the Beige Book corpus published in month t, and $n_t^{Positive}$ represents the total number of positive sentences in the corpus published in month t. The sentiment measure in equation (1) is bounded by -1 and 1, with higher values indicating more positive sentiment. We use tone as a synonym for sentiment, as in papers such as Thorsrud (2020).

Following Barsky and Sims (2012) and Angeletos, Collard, and Dellas (2018), our measure of sentiment in equation (1) can capture both fundamental information (news) about the current and future economy or reflect a causal channel from sentiment (Keynes’ animal spirits) to macroeconomic outcomes as optimism and pessimism rise and fall. We neither attempt to disentangle these two contrasting interpretations of sentiment nor establish causality.⁵ Instead, our focus is establishing (reduced-form) statistical facts about District-level and national sentiment in the Beige Book and examining their predictive power for US recessions. In so doing, our sentiment measure is an aggregated measure in the sense that it quantifies sentiment across different areas of the economy, such as overall economic activity, labor markets, prices, and different sectors.

Our textual analysis focuses on using FinBERT to measure national and District-level sentiment in the 468 successive Beige Books starting from May 1970 through March 2024 that comprise the entire historical database. We access the Beige Book back to 1970 from the public archive maintained by the Federal Reserve Bank of Minneapolis. Over this sample period, the number of words in the Beige Book has increased. The average number of words in the Beige Book is around 1,300. The Beige Book provides a description of economic activity in each of the 12 Districts, as prepared by the respective regional Reserve Bank, and a national summary that is prepared by one of the regional Reserve Banks on a rotating basis. Each of these District-level and national reports starts with a summary of economic conditions before turning to a more specific discussion of economic conditions, including in the labor market, wages and prices, and specific industries of importance in that District.

Features of National and District-Level Sentiment

Figure 1 plots our FinBERT-based measure of national sentiment as extracted from the national summary in each successive Beige Book from 1970 through the latest Beige Book. Alongside this, we plot a shaded area capturing the upper and lower boundaries of the data ranges capturing the 12 measures of District-level sentiment extracted from each District report in the Beige Book. Figure 1 also plots the equal-weighted average of sentiment in each District, an average which we call “consensus” sentiment. In macroeconomic forecasting, it is common to refer to equal-weighted averages of different forecasts as consensus forecasts, and so we adopt this nomenclature here.

We emphasize two key features of Figure 1.⁶ First, focusing on the national sentiment index (in red) we see a clear relationship between rises and falls in sentiment and business cycle turning points for the United States as a whole, as identified by the National Bureau of Economic Research (NBER) and marked as vertical gray bars in Figure 1. As expected, national sentiment falls in recessions.⁷ But Figure 1 reveals that sentiment did not drop as deeply in the Gulf War recession of July 1990 to March 1991 or the dotcom recession of March 2001 to November 2001. This finding fits with the quantitative fact that both of these recessions were characterized by smaller peak-to-trough declines in GDP and smaller rises in the unemployment rate than in the other recessions since 1973. The December 1969 to November 1970 recession at the beginning of our sample was also relatively mild.

Second, the considerable heterogeneity in economic sentiment across Districts characterized by the wide shaded area in Figure 1 is consistent with research (Owyang, Piger, and Wall, 2005; Owyang, Rapach, and Wall, 2009) showing that states both differ in the levels of growth that they experience in recessions and expansion phases of the business cycle and that individual states and Districts are often out of sync with the national business cycle. However, the proportion of Districts that experience negative economic sentiment clearly rises during US recessions. Over the sample in Figure 1, of the 64 months in which the United States was in recession, according to the NBER, 27 of these were associated with sentiment’s simultaneously being negative in all 12 Districts.

We also observe from Figure 1 that national economic sentiment does not always lie in the middle of the District-level estimates, as one might expect if national sentiment were the equal-weighted aggregation of regional sentiment. In fact, since 2021 national sentiment has been more positive than in most individual Districts. That is, the Beige Book narrative about the national economy as a whole has been more upbeat than in the average District, as represented in Figure 1 by consensus sentiment. These differences between consensus and national economic sentiment in Figure 1 are statistically significant.⁸

To test whether, when characterizing national economic sentiment, the Beige Book, intentionally or otherwise, pays more attention to sentiment in some Districts than others, we let the data speak, as it were: We estimate the combination weights on the 12 measures of District-level sentiment that best explain (under squared error loss) movements in national sentiment by regressing national sentiment on the 12 District-level sentiment indices. Table 1, column (1) shows that, on average over our sample from 1970 through 2024, national sentiment loads more heavily on sentiment in specific Districts.⁹

We might expect that the differential weights on the 12 Districts reflect the relative size of each District, if when writing the national narrative the authors implicitly or explicitly acknowledge a District's size. The difference between national and consensus sentiment, as seen in Figure 1, already suggests that Districts do not appear to be treated equally. As one measure of size, we look at the population of each District. The Federal Reserve Bank of San Francisco provides population estimates for the 12 Districts, discussing how the population shares have changed over time.¹⁰ To pick a date reasonably close to the middle of our sample, in 2000 San Francisco was the largest Federal Reserve District, with a population share (relative to the US resident population) of 19 percent, followed by Atlanta with 15 percent, Chicago with 13 percent, Richmond with 10 percent, and Dallas with 8 percent. Supportive of the national narrative’s reflecting the greater population size of these Districts, all five of these Districts receive positive and statistically significant weights in Table 1. In contrast, the smallest four Districts by population in 2000 were Cleveland, Kansas City, and St. Louis, all with 4 percent each, and Minneapolis with 3 percent of the US population. Of these four Districts, only Kansas City is statistically significant in Table 1, column (1). Of the midsized Districts—Boston with 5 percent, New York with 7 percent, and Philadelphia with 8 percent of the population share—Boston and New York are statistically significant, but Philadelphia is not. The fact that the estimated weights on Cleveland, Minneapolis, Philadelphia, and St. Louis are all statistically insignificant from zero, at the 1 percent, 5 percent, and 10 percent levels, indicates that, on average over our sample, these states are under-weighted in the national narrative in the Beige Book.

It is also noteworthy that the R-squared estimate from Table 1, column (1) indicates that 30 percent of the variation in national sentiment is not explained by the typical movements in District sentiment. National economic sentiment is more than a stable linear aggregation of District-level economic sentiment.

Predictive Power of District-Level Sentiment

To measure the information content of the economic sentiment indices in Figure 1 and the possibly differential informational content of national and District-level sentiment, we test their predictive ability for the US business cycle. We focus on analyzing the predictive power of both the national and District-level sentiment indices for US recessions as identified by the NBER on a monthly basis. The NBER maintains a chronology of US business cycles.¹¹ The chronology identifies the dates of peaks and troughs that define economic recessions and expansions. Recessions are defined retrospectively as broad-based declines in economic activity spread across the economy that last for more than a few months.

The forecasting problem that we consider is to predict, using data on each District's sentiment from the Beige Book released in month t, whether the US economy will be in a recession h Beige Book releases into the future, the timing of which we will refer to as simply month t+h. This does mean that we are abusing the definition of a “month” in these predictive regressions, since, strictly speaking, our h-month-ahead forecasts are h-Beige Book-ahead forecasts.¹² As emphasized by Armesto et al. (2009), the Beige Book does not have a regular release schedule. When relating to external economic data, this irregularity can raise questions about how to average Beige Book information to correspond with the period (say, monthly or quarterly) covered by the economic data release. In our application, given our use of monthly NBER business cycle data, we simply relate the Beige Book in a given month to the NBER data at that month, or leads and lags, and therefore neither control for when in the month the Beige Book was published nor for the unequal temporal gap between each of the eight Beige Books typically published per year.

We accordingly estimate h-month-ahead predictive (binary) regressions seeking to explain NBER-dated US expansions in month t+h with District- and national-level sentiment data. Given that the NBER tends to publish its recession dates at a lag, often many months after the beginning of a recession, it remains of interest to backcast (h<0) expansions/recessions and to nowcast (h=0) and forecast (h>0) future ones. Indeed, it is an empirical question whether District-level economic sentiment, as quantified via FinBERT, is a forward-, coincident-, or backward-looking indicator of the US business cycle. The advantage of our in-sample tests of predictive power is that they benefit from longer samples. But, as is well known, there is no reason to expect any in-sample relationship to hold out of sample. This said, our sentiment measures do benefit from being computed in real time and so do not themselves suffer from look-ahead biases.

Table 1, columns (2) through (6) examine the predictive ability of the District-level and national sentiment indices for US business cycle phases (expansions and contractions) as captured by NBER-dated turning points. Looking at the (pseudo) R-squared metrics first that provide a measure of goodness of fit, we see that the predictive power of the sentiment indices is highest at h=0, namely as an indicator of the current business cycle phase. We repeat that, given that the NBER tends to date a recession up to a year after it has started, this result is consistent with the sentiment indices’ still having meaningful predictive power. While, as expected, we see this predictive power drop off as we look further into the future (or back into the past), sentiment retains statistically significant information useful for dating the business cycle. We emphasize that sentiment measures from the Beige Book can be constructed on a timely basis in real time given that, unlike traditional quantitative variables, such as GDP, the Beige Book is published more rapidly to feed into successive FOMC meetings.

We now turn to the estimated coefficients in the predictive regressions for NBER expansions seen in Table 1. These coefficients indicate the weight that should be attached to each District when constructing a forecast of the national business cycle. Positive coefficients indicate that as sentiment rises, there is a higher probability of being in an expansionary phase of the business cycle. Analysis of these coefficients also provides a means of testing whether District-level sentiment provides useful information over and above that contained in the national economic summary. The p-values reported at the bottom of Table 1 provide a direct test of whether national or District-level sentiment indicators contain statistically useful information for explaining US business cycle dynamics. Looking at these p-values, and the associated estimated coefficients, we see that national economic sentiment is not statistically significant at h=0 or when looking into the future (h=3 and h=6). In contrast, our estimates of District-level economic sentiment offer value-added relative to national sentiment. The null hypothesis that District-level sentiment is uninformative is always rejected, with p-values of 0.00.¹³ Inspecting the estimated coefficients on the different Federal Reserve Districts at h=0, we see that sentiment in Chicago, Minneapolis, Philadelphia, Richmond, and San Francisco is most helpful in predicting current US business cycle turning points. The sign on these estimated coefficients is always positive, indicating that as sentiment improves in each District, there is a higher chance of the US economy’s being in an expansionary phase. Boston then becomes the most informative District both three- and six-months ahead, indicating that the most informative District(s) can vary by forecast horizon.¹⁴ In summary, the observed heterogeneity in District-level economic sentiment can be used, over and above the information contained in national economic sentiment, to better forecast US recessions.

Finally, Figure 2 uses the logit model from Table 1, column (4) to plot the conditional probability that the US economy is in recession in the month captured by that Beige Book publication. We see that historically these model-based probabilities match up well with the months that the NBER subsequently defined as recessions. Higher probabilities in Figure 2 are associated with NBER-defined recessions: when the model predicts a recession probability of greater than 80 percent or so, there is a near certainty that the NBER subsequently declared the United States to have indeed been in a recession. From the mid-1980s, and the start of a period of economic stability often called the “Great Moderation” by economists, until the onset of the COVID-19 pandemic in March 2020, we also observe no instances of false alarms, that is, of recession probabilities’ spiking above around 40 percent without a recession actually taking place. Since COVID-19 we have seen, similarly to pre-Great Moderation, much more volatility in recession probabilities. As of March 2024, the probability of a national recession was low, conditioning on the national and District-level sentiment data as extracted from the March 2024 Beige Book. But this probability has bounced around considerably since the pandemic-induced recession of early 2020.

Conclusion

This Economic Commentary shows how natural language processing methods can be used to automate the quantification of text-based regional economic intelligence. Our analysis reveals meaningful economic differences across Federal Reserve Districts in their economic sentiment indices. It also documents that national economic sentiment, as measured by the Beige Book's own national economic summary, is not always the equal-weighted aggregation of District-level economic sentiment. Since the pandemic, national economic sentiment has tended to paint a rosier picture than that experienced in many individual Districts based on our model’s processing of the textual data.

We validate the use of text-based estimates of regional economic sentiment from the Beige Book by showing that they contain statistically useful information about US business cycle phases, opening up their routine use in (real time) nowcasting and forecasting models. Importantly, District-level sentiment contains information useful for identifying US business cycle phases over and above the information contained in the Beige Book's national economic summary. We view these findings as supporting the attention that the regional Reserve Banks give to firsthand reports of economic activity in their respective Districts.