How Well Does the Cleveland Fed’s Systemic Risk Indicator Predict Stress?
A number of financial stress measures were developed after the financial crisis of 2007–2009 in the hope that they could provide regulators with advance warning of conditions that might warrant a corrective response. The Cleveland Fed’s systemic risk indicator is one such measure. This Commentary provides a review of the SRI’s performance from 2001 to 2020 and finds that it has performed well, providing a reliable, valid, and timely signal of elevated levels of financial system stress.
An important tool for monitoring financial stability is a stress index, a constructed measure that indicates whether the banking or financial sectors are “under stress.” A number of such indexes were developed after the financial crisis of 2007–2009, and their primary purpose is to alert the attention of policymakers to conditions that may warrant taking actions. How well these indexes perform can be evaluated along several dimensions. An analogy is a sophisticated burglar alarm system: A good system is reliable in that it will give a clear signal when attention is needed. It is timely in that it will give enough advance notice so that corrective action can be taken. It is straightforward in that the alarm buzzes to get the observers’ attention and then activates other systems to assist observers in making decisions about what actions to take. A good system is also ongoing in that it continues to function during a break-in and will turn off once the threat has disappeared (while retaining a record of when the attack occurred). Finally, the alarm gives valid signals in that it rarely gives a warning of an attack if an attack has not happened.
This Commentary provides a retrospective, narrative analysis of one such index, the Cleveland Fed’s systemic risk indicator (SRI), as a tool for monitoring stress in the financial sector. My approach is to examine the SRI’s daily index values from 2001 to 2020, a period that predates and includes the financial crisis, to see how well it signaled times of known stress along the dimensions above. I find that the index is reliable, timely, and valid. It signaled each of the stressful periods that occurred over the period while signaling only five “false positives,” times when the index predicted stress that did not materialize. Furthermore, those false signals lasted only one day, with one exception that was two days, after which the index went back to normal. Finally, it provided good ongoing information during the financial crisis.
Definition of the Indicator
The SRI is centered on the distance to default (DD), a measure developed by Merton (1974) for firms such as banks that are highly leveraged.1 This measure is based on a subtle difference between what we measure as value in terms of the price of a bank’s stock (the “value of its equity”) and the value of the bank’s total assets (the “value of its assets”). These differ because equity holders get a return on their assets only after the loans are paid off. If the bank cannot manage to pay off its loans, then it is in default and the equity holders get nothing. Because of this, the value of equity is lower than the value of the assets of the bank.
In Merton’s stylized model, a “default barrier” (D) is defined for each firm as the particular value of the firm’s assets in a future time, T, below which the firm will no longer be able to service its debt in full, the firm will be able to pay its equity holders nothing, and the firm will have to use its full assets to pay off its lenders. D is defined by Merton (as well as much of the subsequent literature that uses distance to default) as short-term debt plus one-half of long-term debt, a number that can be computed from bank balance sheets. Distance to default (DD) defines the number of standard deviations that the firm’s value is from D in the context of a stylized model. The distance to default of a single bank (DDi) is computed from the value of the bank’s assets, the value of its equity, volatility values for the bank’s assets and equities (indicators of uncertainty about the future value of those assets and equities), and yields on US Treasuries.
It is important to note that the DD should not be taken too literally as a measure of the probability of default. It is an indicator rather than an exact measure of the default probability for reasons that include the fact that the default barrier is somewhat arbitrary and does not include regulatory capital measures that might shut a bank down. In spite of this, the DD measure that we define is commonly used in risk analysis, portfolio analysis, and formal regulatory discussions of the rating of institutions. It is a concept that is quite familiar within the financial profession.
The SRI builds upon Merton’s DD by calculating the DD for a portfolio of banks, and for each of the individual banks that make up that portfolio. The SRI first calculates the DD for the portfolio of banks using the KBE index, which is an index traded on the Nasdaq exchange. It currently includes 84 banks, each of which is weighted so that the KBE index represents an average price for the banking sector. Because the index is heavily traded, options are sold based upon it, which allows one to calculate the volatility of the portfolio from the options, and from this a “portfolio distance to default” (PDD). The individual DDi’s for all included banks are then averaged with the same set of weights that is used for the KBE index to form an average distance to default (ADD). The SRI is formed by comparing these two numbers.
While individually the quantities PDD and ADD might seem to be good stress indicators, they do not perform exceptionally well as stress indicators on their own. They both embody leverage, balance-sheet strength, and volatility, but these are not sufficient to define triggers that show when regulators should pay attention to stress in the financial sector as a whole. PDD and ADD do marginally better than other indices, such as the average value of banking stocks, in indicating systemic financial stress.
The PDD’s and ADD’s strengths and weaknesses as systemic indicators on their own are due in part to the factors that drive each bank’s DD. A major driver of the levels of each bank’s DD on a day-to-day basis is the value of the bank as measured through the joint equation that calculates the bank’s asset value from its equity price and the bank’s asset volatility. These levels change with the equity prices, and to some extent, the volatility of the bank equity market. As a measure of financial stress, the levels of the averages of these DDi’s share, to some extent, the same disadvantage that a composite indicator of a financial equity index and a volatility index of financial equities would have. Namely, their indication of financial stress would often be driven by factors affecting the economy at large, especially in their false positives.
However, the relationship between the PDD and the ADD does better at capturing common shocks that affect the entire industry. The SRI calculates this relationship as the difference between the two (ADD − PDD). This measure shares some of the good points of the ADD and PDD in predicting stress. If both the ADD and PDD are reduced by a common stress factor, then their difference (ADD − PDD) will be closer to zero. However, in many of the periods of financial stress examined here, including events surrounding the Bear–Stearns failure, the difference was more timely in predicting the seriousness of the event. Precisely why this is so is the subject of ongoing research. As defined, the SRI is usually a positive number and decreases with stress.
How Well Has the SRI Performed in the Past?
A look at the historical values of the SRI in figure 1 indicates that during periods of intense stress, the level of the index falls below 0.1 and then rises to 0.5 and stays at this level for some period of time. So I choose a trigger value of 0.1 to examine how well the index performs in the almost 20 years for which we have data. The way that I assess this record is by examining how well the SRI has anticipated known periods of stress since February 2001 considering two questions: Were there any false positives during this period, and how soon, as a stressful period was developing, did the SRI alarm go off? I argue that the SRI with a trigger point of 0.1 has an excellent track record as an indicator of financial-sector stress for the entire period.
This exploration is made somewhat difficult by the fact that defining periods of financial stress is subjective.2 For the purposes of this Commentary, I define them as periods that have been named in the financial literature by pundits, academics, and financial columnists. In some cases, the name simply is a label of the assumed cause of the financial stress.
I have identified 15 subperiods of stress, listed in table 1. These are also shown in a timeline in figure 1 along with the SRI. Because the SRI is a daily measure and possible crisis events unfold slowly, several of the stress periods are listed multiple times in figure 1, with the dates of various subcrisis events during the crisis denoted. In all, there are nine separate events.
|7/30/2002||Sarbanes–Oxley Act||Regulatory civil and federal accounting law passed in response to Enron and WorldCom accounting scandals|
|6/20/2007||Bear-Stearns 1||Two Bear-Stearns mortgage-focused hedge funds collapse|
|3/14/2008||Bear–Stearns 2||The Federal Reserve Bank of New York makes a $25 billion loan to facilitate the JP Morgan purchase of Bear-Stearns|
|9/12/2008||Lehman Brothers 1||Friday before Lehman Brothers fails|
|9/15/2008||Lehman Brothers 2||Lehman Brothers declares bankruptcy|
|9/16/2008||Lehman Brothers 3||Reserve Primary Fund (a money market fund) “breaks the buck,” and the US government lends AIG $85 billion|
|4/18/2011||US downgrade and euro debt 1||S&P warns that US debt might be downgraded if the debt ceiling is reached and long-term fiscal plans are not made|
|8/5/2011||US downgrade and euro debt 2||S&P downgrades US debt from AAA to AA+|
|8/24/2015||Chinese sell-off 1||Shanghai Composite index falls 8.48 percent|
|1/4/2016||Chinese sell-off 2||Shanghai Composite Index falls 6.9 percent|
|1/7/2016||Chinese sell-off 3||Shanghai Composite index falls 7 percent within 30 minutes of opening|
|6/24/2016||Brexit||Day after the British voted to exit the European Union|
|6/1/2018||Italian sovereign debt crisis||New Italian coalition government is formed; Italian bond yields steadily rise|
|9/16/2020||Repo disruption||The repo market experiences a shortage of liquidity, causing rates in US short-term funding markets to spike as high as 5 percent|
|2/25/2020||COVID-19||The S&P 500 falls 3.03 percent in response to the COVID-19 news along with a large oil price shock, starting a long period of high volatility in the market along with a general decline in the indices|
Table 2 shows the behavior of the indicator for the same time frame. Its track record is remarkably good. Not only does it catch the major episodes of financial stress, it also anticipates them with a remarkable foresight. The Sarbanes–Oxley Act is anticipated on the day of the first real discussion leading up to it, six months before its passage. Although it was widely anticipated to give the financial sector trouble with its complication of capital requirements during a time of slow growth, in the end the difficulty for the financial sector did not pan out. However, the SRI detected this stress on the day that all the high-ranking officials of the White House and Federal Reserve Chair Alan Greenspan agreed that the Enron scandal necessitated a thorough accounting reform bill.
On February 22, 2002, nearly 6 months before passage of act, when the passage of some act of reform was thought to be likely
|The SRI plummeted to rare negative levels on the day that Alan Greenspan, Paul O’Neill, Glenn Hubbard, and other high-ranking officials met to discuss the need for accounting reform in light of the Enron scandal. The index remained low, indicating a high level of stress, until it became clear that the implications of the new law for bank capital would not affect financial-sector stability, at which point the stress indicator waxed fairly quickly, indicating low levels of stress. A slow economic recovery led to weaker bank balance sheets than usual, so the index remained somewhat low until April 2003.|
|March 22, 2004||1||Barney Frank announced that predatory lending regulations should apply to national banks. The SRI returned to normal the next day.|
|June 13–14, 2006||1||The US House Committee on Financial Services called a hearing on Basel II’s final capital requirements, suggesting the possibility of adoption by the United States. After two days, it was clearer that the requirements would not be adopted quickly on the whole by the United States, and the SRI returned to normal.|
|July 18, 2006||1||Citicorp announced that strong investment banking had boosted its profits 3.6 percent. The SRI returned to normal the next day.|
On July 23, 2007
|The first sign that Bear–Stearns was in trouble came with the collapse of two of its mortgage-based hedge funds on June 20, 2007. The SRI fell to a very low level of 0.06 on July 23, 2007, and remained subdued throughout the financial meltdown. The low level of the SRI preceded any expression of alarm in the financial press where, at this time, the Bear–Stearns trouble was seen as local and containable.|
On February 28, 2008
|The low level of the SRI attenuated for less than a week and then returned to its reduced level, suggesting that the index recognized that Bear–Stearns was representative of a more systemic financial malaise.|
|Lehman Brothers 1,2,3||Yes.
On September 12, 2008
|Markets were roiled on this day, the Friday before the Monday on which the Fed announced it would not bail Lehman out. The SRI, which had generally been in the 0.5 or below range, dove to nearly zero, suggesting that the index anticipated the seriousness of the Lehman troubles.|
Continuous monitoring July 23, 2007, until September 2012
|The SRI stayed between 0.5 and zero from 2007 until September 2012. As the economy finally began to experience some robust growth rates in the first quarter of 2012 (though these declined temporarily in the third quarter of 2012), the financial sector also began to get healthier in 2012.|
|December 19 and 24, 2014||2||On December 19 the SRI briefly trenched because of very high market growth, and on December 24 it fell and rose again because options traded thinly. The index corrected on the next day after each episode.|
|Chinese sell-off and Brexit||Yes.
Continuous monitoring December 2014 to summer 2015
|The SRI fell slightly to within a range between 1.0 and 0.3 (except for the brief false positives in the events above, where it shot down to zero) in February 2015, after which it rose to higher levels. Financial markets had determined that the exposure of the US financial system’s stability to global forces is small.|
|Italian sovereign debt crisis||Not seen as important after experience with Brexit and the Chinese selloff. The SRI was generally above 1.|
|Repo disruption||September 2019||The index did not fall, as markets saw this as a short-term disruption.|
|COVID-19||February–March 2020||The index was only moderately depressed because, while the oil shock and the worries about the recession affected individual banks, the financial system as a whole was not seen as endangered because of any structural weakness.|
The Bear–Stearns problem is seen by the index as a major systemic problem well in advance of the time at which the financial press thought that it would be a problem. Also of note is the Lehman bankruptcy, which not only was associated with an index value indicating a highly elevated level of stress but was also presciently ahead of the curve in that the index shrank to negative levels on the Friday before the rest of the markets were roiled by the Fed’s announcement that there would be no bailout. Further, the index also seems to learn from past experience. Increasing globalization caused Brexit and the Chinese selloff to send the index lower. However, these two experiences seemed to “teach” the index that financial-sector stability was less exposed to global events, so that the Italian debt crisis did not affect it nearly as much.
I counted five false-positive events, which the SRI cleared up quickly: Four of them lasted one day on the index’s radar, and one lasted two days. In each case, I could find some reason for the false positive. On three of the days, a large positive outcome in the equity markets gave a common shock that was picked up by the stress index. In two of the cases, all that I could find on that date was a regulatory announcement that might have caused the stress, along with higher-than-average volatility of bank equity prices. In each of these cases, neither the common shock nor the news was enough to sustain the index’s indication of high stress for more than two days. At five, the number of false positives over a 20-year time span is small.
The overall rule from the historical experience of the index is this: When the difference between ADD and PDD is less than 0.1 for more than a couple of days, it has always been verified to represent an underlying stress that represented a risk to the stability of the banking sector. The periods of large stress, such as happened with the financial crisis of 2007–2009, are characterized by long periods during which the indicator seldom goes above 0.5. In this sense, the indicator stays “on” while the financial sector is still stressed.
The lack of false positives in this indicator can be compared with the performance of simpler indices that might be used to show financial stress. For example, a component of the SRI, bank equity volatility, could be an indicator of stress, showing as it does, uncertainty about the future value of banks. However, whatever reasonable threshold value of the volatility that one chooses, banks go through many sustained periods of high volatility that represent very little stress to the financial system, but rather uncertainty about future earnings. This can occur when the structure of the system is sound, and indeed, can be the result of banks feeling confident enough in the system that they underwrite more profitable yet riskier projects. The SRI distinguishes between these periods and actual stress periods quite well, so that these common shocks to the system that increase profitability will register in the SRI after only a day, but then the index returns to normal levels. However, when the SRI has been sustained below a level of 0.1, it has always indicated stress, and it has signaled that stress early.
A good financial-stress indicator is reliable, timely, straightforward, valid, and ongoing. The SRI has demonstrated most of these qualities. For reliability and timeliness, we see it has provided an early signal of every period of financial stress throughout the past 20 years, and in the largest crisis during the period it indicated a major problem even before the Bear–Stearns bailout and continued in high alarm mode throughout the financial crisis. The SRI is straightforward to interpret in that any time the indicator reaches 0.1 and stays lower than 0.5 for more than a few days, it indicates that the financial sector is seriously stressed. The index provides an ongoing signal of a stress period, remaining at a depressed level throughout such periods. It is valid in that it has a small number of false alarms in its history, and in all of these cases, the false alarm went away after a day or two.
The reason that the SRI works well is that it combines measures of balance-sheet strength, volatility, and correlation of the asset values of the major banks with the forward-looking characteristics of option prices. A possible disadvantage of the indicator is that it is terribly complex, so that it is sometimes difficult to disentangle why it works in the way it does during a period of financial stress. Further, in its current form, it does not single out which banks are responsible for a crisis (although this is in principle possible). The fact that the indicator raises the alarm when balance sheets are weak and banks are highly leveraged, along with the fact that information hitting the financial sector is both highly volatile and highly correlated, means that false positives are minimized. This minimization is balanced by the fact that the information is acquired from option values that are forward-looking and thus the indicator incorporates the latest information, so that when a financial crisis is real, it trips the SRI warning early.
The SRI is one of many possible indicators that could be used. Future research will study how well the index performs compared to these other methods and where it outperforms or underperforms them. For the time being, however, there is much that recommends the SRI as an early indicator of financial stress.
The views authors express in Economic Commentary are theirs and not necessarily those of the Federal Reserve Bank of Cleveland or the Board of Governors of the Federal Reserve System. The series editor is Tasia Hane. This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. This paper and its data are subject to revision; please visit clevelandfed.org for updates.
- The technical details involved in Merton’s approach and that of the SRI are presented in the online appendix. Return to 1
- One could, of course, rely on world lists of financial crises, but this turns out to be problematic, particularly for the twenty-first century, which is where we have data needed to construct our indicator. One of the few available lists gives only one crisis for this period, which starts with the bailing out of Bear–Stearns and continues until 2016, when the series ends. Return to 2
For Further Reading
- Merton, Robert C. “On the Pricing of Corporate Debt: The Risk Structure of Interest Rates.” 1974. Journal of Finance, 29(2), 449-470. https://doi.org/10.2307/2978814.
- Saldias-Zambrana, Martin. 2013. “Systemic Risk Analysis Using Forward-Looking Distance to Default Series.” Journal of Financial Stability, 9(4): 498-517. https://doi.org/10.1016/j.jfs.2013.07.003.
Craig, Ben R. 2020. “How Well Does the Cleveland Fed’s Systemic Risk Indicator Predict Stress?” Federal Reserve Bank of Cleveland, Economic Commentary 2020-28. https://doi.org/10.26509/frbc-ec-202028