The Zen of IFRS 9 Modeling
The Zen of IFRS 9 Modeling
At Open Risk we are firm believers in balancing art and science when developing quantitative risk tools. The introduction of the IFRS 9 and CECL accounting frameworks for reporting credit sensitive financial instruments is a massive new worldwide initiative that relies in no small part on quantitative models. The scope and depth of the program in comparison with previous similar efforts (e.g. Basel II) suggests that much can go wrong and it will take considerable time, iterations, communication and training to develop a mature toolkit that is fit-for-purpose.
One of our signature blog posts (now part of the Open Risk Manual) is the Zen of Modeling, a collection of short aphorisms that aim to prompt quantitative analysts (also known as modelers, machine learning practitioners etc.) to think deeper about the nature of their work and in particular the delicate balance between art and science. In this blog post we take a look at the art and science of IFRS 9 Expected Credit Loss modeling, The Zen of IFRS 9 Modeling!
An undocumented risk model is only a computer program
As the pressures of the IFRS 9 implementation deadline loom large, everybody will produce an estimate (by hook or crook). The tools used in the process will span the range of sophistication, from back-of-the envelope calculations to massive IT systems. Yet whether the calculation is a coherent framework will largely be traceable to the amount and quality of the documentation that supports and explains the ECL measurement framework.
A risk model that cannot be programmed is only a concept
The IFRS 9 standard is not prescriptive, so many of the implementation details involve a large element of discretion. Such discretion can quickly degenerate into untraceable, non-reproducible estimates - in other words, absence of provenance. Documentation is a important mechanism (See previous koan) to contain this phenomenon. Where applicable, forcing the quantification of subjectivity e.g. via expert based models can further enhance pinning down the more subjective parts of the methodology.
A risk model only comes to life with empirical validation
Reporting under IFRS 9 is not a one-of exercise but a recurring task. External stakeholders will have access to both panel data (range of reports from different firms) and timeseries data (subsequent reports from the same firm). Internal IFRS 9 ECL models must be validated using e.g., historical data.
Correct implementation of an imperfect model is better than wrong implementation of a perfect model
The information and modeling requirements of IFRS 9 are more complex than even the most advanced internal ratings based risk models of the Basel framework. The implementation of complex models carries significant intrinsic execution risk, which is even higher if the organization is not accustomed to using such tools. Focusing on internalizing a simpler framework and then iterating as experience accumulates is one strategy to mitigate this risk.
In complex systems there is always more than one path to a risk model
The scope of reporting IFRS 9 Loss Allowance spans a significant fraction of the world’s credit assets and at a very granular level. It stands to reason that both conceptually and in practice there are multiple ways of achieving desired objectives such as probability-weighted estimate of credit losses. Different approaches may involve different tradeoffs in terms of data requirements, embedded assumptions, ability to interpret etc. With active research and exchange of information preferred paths might gradually emerge - but it is very unlikely that there is a unique one-approach-fits-all solution.
There are no persistently true models but there are many persistently wrong models
IFRS 9 requires to produce an unbiased estimate of expected credit losses on an ongoing basis. Given the constant evolution and complexity of underlying economies and the difficulty of collecting relevant data it would be only by accident if a model was found to be achieving this objective over longer time spans. The objective is instead to avoid being obviously wrong (=biased).
Correlation is imperfectly correlated with causation
Especially since the financial crisis the use of macro-economic stress testing has dominated the thinking and practice of banking professionals. A core element of that practice is establishing correlations of portfolio performance with macro-economic factors. The expectation of using macro-economic scenarios in IFRS 9 as Forward Looking Information is quite firm. Yet such correlations may be flimsy and inadequate in capturing the full range of volatility affecting financial instruments.
Nirvana is the simplest model that is fit for purpose
The combined requirements of granularity, probability weighted forward looking scenarios and lifetime ECL assessment means that each produced ECL estimate requires a substantial number of parameters and model assumptions. Ideally these should be weeded down to smallest number necessary. Thus avoiding on the one hand needless complexity and on the other hand deficient simplicity.
Hierarchical systems lead to hierarchical models. Uncertainty is highest at the top
The wide scope of IFRS 9 means that estimates for large portfolios are aggregated from many individual contributions. This creates an accumulation of uncertainty as potentially incompatible assumptions, data sets and forward looking views are combined. The combination of the bottom-up and top-down paths is one strategy for mitigating this model risk.
Model assumptions are more vulnerable than model structure
The analytical toolkit of multi-period credit risk modeling is rather firmly established. Yet any concrete implementation of a particular model structure requires a substantial number of assumptions, in combination with the available evidence. There is usually more model risk in those assumptions than in the selection of overall analytic framework.
Building risk models is easy, managing model risk is not
Even after ensuring documentation of assumptions, transparent implementation, consistency across portfolios there is residual model risk. For example different analytic frameworks may adopt different distributional properties that might be difficult to validate empirically. The implementation of alternative models provides a basic estimate of the possible discrepancy by such choices.
Models inherit their nature from their creators, but nurture from their use environment
The life of an IFRS 9 ECL model does not stop at implementation, it begins there. Successful models will enable users to internalize the model structure and interpret unfolding economic developments in terms of model parameters and model outcomes. Overrides and empirical validation close the feedback loop that further tunes the model to the reality it aims to approximate.
Models don't speak people's languages. It is the responsibility of the modeler to translate to an understandable idiom
While there is much debate about model complexity in finance, some of it is misguided. The principles underlying IFRS 9 ECL modelling are fairly basic probability. Modelers must lay out the path for non-technical people to reach the essence of the framework.
People don't care about models, only about model outcomes. It is the responsibility of the modeler to be responsible
The combination of subjectivity and visibility of model outcomes means that reported IFRS 9 model outcomes will be heavily managed. Modelers are responsible to protect the integrity of their methodologies. This is achieved by constructing frameworks that incorporate subjectivity in a coherent and documented manner.
Models closed in black boxes perish. Models live a healthy life when open and free
IFRS 9 models are ultimately algorithms and software. We know from many other fields that Open Source is the dominant paradigm for reliable advancement of such tools
Feedback
What is your experience of the art and science of IFRS 9 Modeling? Give us your feedback using the button on the left!