What is the future of stress testing?
To speculate on the future of Stress Testing we need first a basic definition what is stress testing. Broadly speaking, the goal of Stress Testing is to assess how a system would behave under adverse conditions that - while not the most likely outcome with the knowledge of today - are within the realm of the plausible. There are, broadly speaking, two types of stress testing: The Real type and Hypothetical version. Let us start with the real stuff.
Real Stress Testing:
Real stress testing is applicable to systems where representative copies of what we want to stress test are readily available and/or dispensable. Think of machinery, tools, equipment, automobiles, etc. One picks a representative sample (a particular car), smashes it against an immovable obstacle and has immediate feedback on how a similar instance would behave in, say, a real life car accident.
Thus, real stress testing provides an empirical answer to a “What If” question that captures a plausible future scenario. This type of stress testing subjects a copy of our system to the actual stress event conditions (e.g. increased temperature or pressure) and monitors / measures the impact and outcomes of the test.
Hypothetical (Scenario based) Stress Testing
For systems where representative copies are not readily available and/or are not dispensable, stress testing is performed via analyses that construct an estimated response of the system to hypothetical scenarios. Thus, hypothetical stress testing is, in all cases, a simulation.
In scenario based stress testing we start with what we think is an adequately faithful representation of the current state of our system. We subject this to what we think are adequately representative stress scenarios. We then derive outcomes on the basis of what we think is an adequately faithful representation of the intrinsic evolution of the system from its current state to the stressed state
In the context of bank risk management stress testing is clearly a hypothetical exercise. Nobody, e.g., incites a run on a representative bank to identify whether its management has adequate tools to survive the induced crisis. Instead, a stress test translates into a : How would a bank fare if the “economy” took a turn for the worse? Would it be a Going Concern versus Gone Concern ?, that is, would it survive as an operating entity according to the applicable laws and regulations?
Hence, to perform the thought experiment the stress test must in sequence:
- define what a bank is (for stress testing purposes) in terms of provided information (accounts etc.)
- define what is an economy (for stress testing purposes) and how it “might go worse”
- evaluate how the bank will fare under a range of scenarios
If this all sounds like a tall order it is because it is! Hypothetical stress testing of complex systems is nowadays not uncommon, aided primarily by the appearance of the digital computer and the computational power that this entails. Maybe the most weighty such example is the Stockpile stewardship program. This refers to a United States program of reliability testing and maintenance of nuclear weapons that aims to avoid the use of real nuclear testing.
The programme makes use of simulations (using non-nuclear explosives tests and supercomputers, among other methods) and applications of scientific knowledge about physics and chemistry to the specific problems of weapons aging (and subsequent performance of a nuclear device if detonated).
The story of the Vasa
Stress Testing in banking and finance is actually an old and venerated risk management tool and in simpler incarnations was used long before Quantitative Risk Management became a dominant paradigm. Stress testing as a term shot to public prominence in particular after the Great Financial Crisis, essentially as a tool to help remedy the weak risk management practices of regulated financial intermediaries (and related gaps in the regulatory frameworks).
In a Keynote speech Andrea Enria, Chair of the Supervisory Board of the ECB, at the European Systemic Risk Board (ESRB) Annual Conference recounted the story of the Vasa as an example of a system that has not been stress tested appropriately:
In 1628, the Vasa – the pride of the Swedish navy – was the most powerful warship in the Baltics. Until she sank about half an hour into her maiden voyage, that is. So what happened? Well, nothing special when it comes to sailing ships: the wind simply picked up.
The problem was that the Vasa’s design was rather unstable. She had been built for the shallow waters around Stockholm, so not too much of the ship was below the waterline and a lot of weight was concentrated in her upper structures. Thus, when the wind picked up, it pushed the ship so far over on its port side that water poured in through open gunports. And that was the end of the Vasa – and a huge embarrassment for the King of Sweden, Gustavus Adolphus.
How does one stress test a ship for sea-worthiness? The answer requires us to dive into millennia of maritime engineering know-how, trial-and-errors, cumulating scientific knowledge culminating nowadays to our ability to simulate entire Maritime systems . It is not really necessary to dive into these details, any more so than we did for nuclear weapons simulations. What is clear, ex-post, is that whatever analysis and stress testing was performed on the Vasa was entirely inadequate. In fact, according to wikipedia we have the following very interesting tidbit:
... the king's subordinates lacked the political courage to openly discuss the ship's problems or to have the maiden voyage postponed. An inquiry was organised by the Swedish Privy Council to find those responsible for the disaster, but in the end no one was punished.
The Lucas Critique
It would be an interesting and fun exercise to explore the analogies of stress testing banks with the simulation of nuclear weapons or ship sea-worthiness, but we can immediately see a major difference. A bank can think for itself. This comes about as the result of a “bank” being the sum total of stakeholders with a variety of objectives, obligations and options which they are more likely than not to exercise if and when deemed necessary.
The implication is that when simulating an economic / financial system as a deterministic entity responding to changing conditions we might be oversimplifying things. This is actually a well known issue in modelling economic systems more generally. The Lucas critique first raised awareness of this problem in the context of macroeconomic modelling. It essentially suggests that simulation models of economic systems that use parameters established historically (which are, nota bene, our only empirical basis) might not be valid in future scenarios unfolding under different policies.
The Lucas critique suggests that if we want to predict the effect of a policy experiment, we should model the “deep parameters” (relating to preferences, technology, and resource constraints) that are assumed to govern individual behavior: so-called “microfoundations.” While the microeconomics of banking is an entire research / academic topic of its own1, incorporating such microfoundations in stress testing might overcomplicate things to the point of incomprehension / un-tractability.
For the time being the inability to capture variable responses during a stress scenario can be seen e.g., in the need for the Static Balance Sheet Assumption which fixes the optionality available to agents in an ad-hoc way. In any case, achieving realism in the stress testing exercise must address in a balanced way all the complications that are important, which, as recent events demonstrate for the n-th time, can be a major challenge if the thought framework underpinning and constructing the simulation habitually and casually excludes important elements.
Covid-19 and the Pretense of Non-Knowledge
When it comes to analysing (modelling or simulating) complex systems such as the economy (and many of its subsets) it is easy to expose oneself to the Pretence of Knowledge critique. Made famous by Hayek, this critique highlights the false certainty that we understand the system sufficiently to be able to formulate policy or take other action in accordance with that understanding. Yet the Covid-19 pandemic indicates there is another legitimate critique that is equally if not more relevant: The pretence of “non-knowledge”.
The pretense of non-knowledge can variably also be termed as “head in the sand”, “Ostrich effect” or ignoring real and present danger. In early 2020, while covid-19 might have been already growing exponentially somewhere in the world, the global collective of policy and risk managers contributing to the global annual survey of the World Economic Forum did not consider a pandemic to be a top-ten risk:
In fact in the decades before, the prospect and risk of a pandemic was very rarely high on the risk manager agenda:
After the onset of the pandemic some voices immediately raised the idea of Covid-19 being a Black Swan , i.e. a risk event that could not have been predicted. That argument quickly subsided as a pandemic is both:
- a recurring Risk Event (it is based on the well entrenched and unavoidable biological basis of our existence.)
- many national / supranational entities around the world are exclusively focused on anticipating / mitigating pandemic risks.
What will be the impact of the Covid-19 pandemic on the theory and practice of stress testing? In the first instance, the pandemic stress tested the stress testing framework itself - and found it wanting, in the sense that the European 2020 exercise had to be postponed. There are two major implications of this outcome that cannot be ignored:
- Stress testing exercises (as they are being performed) consume vital resources that are scarce
- They are not useful when actual stress events happen
As we continue exploring the challenges and opportunities of stress testing anno 2031 we can already state:
The stress testing frameworks of 2031 should be such that in a crisis they are the most important tool rather than the first to be ignored.
The tragedy of the risk horizon
Yet as commentators have pointed out already, the invisibility of Covid-19 and associated biological risks is not the only class of real risks ignored by economic / financial thinking. In fact, the entire nascent sector of Sustainable Finance concerns the recognition and intermediation of economic activity that does not assume away the realities of our existence.
The road towards sustainable finance is long and by 2031 we will have seen a few more turns. The Tragedy of the Risk Horizon denotes a behavioural / economic phenomenon where certain risks may exceed the management (Risk Horizon) of financial / economic actors, imposing a cost on future generations that the current generation has no direct incentive to fix. The phrase was coined by Mark Carney, Governor of the Bank of England and Chairman of the Financial Stability Board, in the context of Climate-Related Risk2
The expression paraphrases the well-known economic phenomenon of the tragedy of the commons, where in a shared-resource system, individual users acting independently according to their own self-interest, behave contrary to the common good of all users. The impact of such Risk Management Failure is aggravated when risks are of cumulative nature, hence where earlier Risk Mitigation would mean less costly adjustment.
In the context of Climate Change, environmental parameters and risks evolve over time horizons that are longer than:
- the business cycle of environmentally impacting sectors (typically less than a decade)
- the political cycle of decision makers (4-5 year)
- the horizon of technocratic authorities, like central banks (with monetary policy having a 2-3 horizon while financial stability policies typically less than a decade)
- the refinancing horizon of corporate liabilities (around 3 years)
- the holding period of financialized / securitised / traded risks which is as short as the available liquidity permits
This brings us to another dimension of 2031, fit-for-purpose, stress testing:
The stress testing frameworks of 2031 should be effective in capturing rather than hiding the actual timescales of risks.
The bank-sovereign nexus, virtuous or vicious and does it matter?
The actual societal response to the Covid-19 crisis, paired with the response to the Great Financial Crisis points out towards another problematic element of stress testing frameworks. In the recent Speech by Isabel Schnabel Member of the Executive Board of the ECB she discussed the very important role that governments have played in the handling of the economic and financial “side-effects” of the pandemic.
While few people would question the wisdom, rationality and outright necessity of public sector intervention in the current circumstances, the exercise has significant collateral damage in the conceptual frameworks underpinning stress testing and financial risk management, namely the notion of an arms-length, private entity that assesses and underwrites business risk - for profit.
As a concrete example: A pandemic has a major impact on mobility risk, the ability of people to go about, to perform (among others) economic activities. The patterns of mobility that we pursue are not happening in vacuo but are explicitly underwritten (e.g. through financing and subsidies for airlines or all other mobility technologies).
As we push further into environmental degradation and concomitant risks it will transpire more and more frequently that un-sustainability and business risk have strong overlap. We mentioned above the Lucas critique as concerning the response to policy changes by economic agents. What about the policy changes themselves? Can those always be assumed? Is the government always going to be coming to an explicit or implicit rescue of the banking / financial system? (effectively creating liabilities for the collective). What happens if the public sector is willing but not able to support the affected business sectors? What if the attribution and “blame game” is a gray area?
The stress testing frameworks of 2031 should be able to navigate the confusing pattern of liabilities emerging out of many decades of unsustainable economic activity.
Digitisation, Decentralization, Shadow Banking and Stress Testing the Crypto-Economy
By 2031 the digital transformation of banking might be actually complete, in the sense that the historical pattern of chronic mal/under-investment in digital technology will have been overcome: the financial system will have been largely digitized, with whatever risks and opportunities this entails.
It is really difficult to envisage with any degree of confidence the form stress testing will take in such a fully digital / decentralized economic system. The current form of banking reflects many centuries of paper based accounting and organizations highly optimised around paper technology. There is enormous flexibility in organizing the flow of information in digital systems. There are for example immense possibilities for much more transparency, more automated assessment and faster corrective measures. On the other hand the manias, bubbles, leveraged speculation that are endemic to primitive, unregulated human economic behavior can reach new levels in a world where everybody can transact with the help of algorithms in microseconds.
In fact we do have a preview of the sort of disruption and risks that emerge through the evolution of financial system technologies. It is the experience and knowledge obtained during the growth of the securitisation era. That episode should serve as a textbook on how information, quantitative models and algorithms, incentives, regulatory blind spots etc. can take any basic financial technology and turn it into a weapon of economic destruction.
The stress testing frameworks of 2031 would hopefully use technology to get us one step closer to addressing the malfunctions and financial system risks of 1931, 1831 and every century before!