Model Validation

Semantic Web Technologies integrate naturally with the worlds of open data science and open source machine learning, empowering better control and management of the risks and opportunities that come with increased digitization and model use

The ongoing and accelerating digitisation of many aspects of social and economic life means the proliferation of data driven/data intermediated decisions and the reliance on quantitative models of various sorts (going under various hashtags such as machine learning, artificial intelligence, data science etc.).

The Risk Function Ontology

The Risk Function Ontology is a framework that aims to represent and categorize knowledge about risk management functions using semantic web information technologies. Codenamed RFO codifies the relationship between the various components of a risk management organization. Individuals, teams or even whole departments tasked with risk management exist in some shape or form in most organizations. The ontology allows the definition of risk management roles in more precise terms, which in turn can be used in a variety of contexts: towards better structured actual job descriptions, more accurate description of internal processes and easier inspection of alignement and consistency with risk taxonomies. See also live version and the white paper OpenRiskWP04_061415.

Semantic Web Technologies

The Risk Model Ontology is a framework that aims to represent and categorize knowledge about risk models using semantic web information technologies.

In principle any semantic technology can be the starting point for a risk model ontology. The Open Risk Manual adopts the W3C’s Web Ontology Language (OWL). OWL is a Semantic Web language designed to represent rich and complex knowledge about things, groups of things, and relations between things. OWL is a computational logic-based language such that knowledge expressed in OWL can be exploited by computer programs, e.g., to verify the consistency of that knowledge or to make implicit knowledge explicit. OWL documents, known as ontologies, can be published in the World Wide Web and may refer to or be referred from other OWL ontologies. OWL is part of the W3C’s Semantic Web technology stack, which includes RDF, RDFS, SPARQL, etc

The motivation for federated credit risk models

Federated learning is a machine learning technique that is receiving increased attention in diverse data driven application domains that have data privacy concerns. The essence of the concept is to train algorithms across decentralized servers, each holding their own local data samples, hence without the need to exchange potentially sensitive information. The construction of a common model is achieved through the exchange of derived data (gradients, parameters, weights etc). This design stands in contrast to traditional model estimation where all data reside (or are brought into one computational environment).

Overview of the Julia-Python-R Universe

A new Open Risk Manual entry offers a side-by-side review of the main open source ecosystems supporting the Data Science domain: Julia, Python, R, sometimes abbreviated as Jupyter.

Motivation

A large component of Quantitative Risk Management relies on data processing and quantitative tools (aka Data Science ). In recent years open source software targeting Data Science finds increased adoption in diverse applications. The overview of the Julia-Python-R Universe article is a side by side comparison of a wide range of aspects of Python, Julia and R language ecosystems.

The challenge with historical credit data

Historical credit data are vital for a host of credit portfolio management activities: Starting with assessment of the performance of different types of credits and all the way to the construction of sophisticated credit risk models. Such is the importance of data inputs that for risk models impacting significant decision-making / external reporting there are even prescribed minimum requirements for the type and quality of necessary historical credit data.

Data Scientists Have No Future

The working definition of a Data Scientist seems to be in the current overheated environment:

doing whatever it takes to get the job done in a digital #tech domain that we have long neglected but which is now coming back to haunt us!

The Zen of IFRS 9 Modeling

At Open Risk we are firm believers in balancing art and science when developing quantitative risk tools. The introduction of the IFRS 9 and CECL accounting frameworks for reporting credit sensitive financial instruments is a massive new worldwide initiative that relies in no small part on quantitative models. The scope and depth of the program in comparison with previous similar efforts (e.g. Basel II) suggests that much can go wrong and it will take considerable time, iterations, communication and training to develop a mature toolkit that is fit-for-purpose.

Guiding principles for a viable open source operational risk model (OSORM)

Such a framework:

1. Must avoid formulaic inclusion of meaningless risk event types (e.g., legal risk created by the firm’s own management decisions) or any risks where the nature and state of current knowledge does not support any meaningful quantification. Such potential risks would be managed outside the framework
2. Must employ a bottom-up design that addresses the risk characteristics of simpler business units first and (if needed) creates a combined profile for a more complex business in a building block fashion.
3. Must avoid formulaic integration between different event risks (that is, positing dependency or correlation) when qualitative and quantitative reasoning does not support it
4. Must cover a comprehensive event risk taxonomy that is well adapted to the nature of the firm
5. Incorporates effectively all relevant organizational information (e.g., Key Risk Indicators or other business line characteristics that are demonstrably generating or affecting operational risk)
6. Involves clear explanatory narratives as to the causes of operational risk (and the drivers of severity). Thus, it links operational risk realizations to concrete attributes of the firm and its environment. This implies preference for fundamental (factor) modeling over agnostic reduced form approaches
7. Is sufficiently concrete and expressive to enable introducing management control parameters. In other words, answering the question: how can we reduce operational risk
8. Must use all relevant empirical historical loss data either at the stage of the model building or at the validation stage
9. The probability distributions employed should reflect the nature of the risk and be valid at the widest possible confidence level range
10. Must incorporate forward looking elements (e.g. via scenarios) that are consistent with the current and past behavior / response of the firm
11. Should allow the documentation any difficult or subtle choices with solid expert reasoning
12. Is the simplest model that is fit for purpose!
13. Last but not least: It is open source, making its advantages and limitations transparent to all stakeholders

Discussion / contributions are welcome.

Save the AMA whale

ΝΒ: This is not a post about real whales and the ongoing struggle to keep these magnificent mammals alive for future generations to marvel at. Hopefully the individuals who have risked their lives to bring the near extinction of many whale species to worldwide attention will not take offense with us usurping imagery linked to this valiant campaign. We simply want to draw attention to another, rather more armchair type of campaign, namely: saving the_AMA risk model. A bit more esoteric as a cause, but ultimately a good cause nevertheless_

Risk modeling is as much art as it is science

The Zen of Modeling aims to capture the struggle for risk modeling beauty

1. An undocumented risk model is only a computer program
2. A risk model that cannot be programmed is only a concept
3. A risk model only comes to life with empirical validation
4. Correct implementation of an imperfect model is better than wrong implementation of a perfect model
5. In complex systems there is always more than one path to a risk model
6. There are no persistently true models but there are many persistently wrong models
7. Correlation is imperfectly correlated with causation
8. Nirvana is the simplest model that is fit for purpose
9. Hierarchical systems lead to hierarchical models. Uncertainty is highest at the top
10. Model assumptions are more vulnerable than model structure
11. Building risk models is easy, managing model risk is not
12. Models inherit their nature from their creators, but nurture from their use environment
13. Models don’t speak people’s languages. It is the responsibility of the modeler to translate to an understandable idiom
14. People don’t care about models, only about model outcomes. It is the responsibility of the modeler to be responsible.
15. Models closed in black boxes perish. Models live a healthy life when open and free.

Update May 2017: The Zen of Modeling has also been integrated into the Open Risk Manual to enable easier linking to the knowledge based developed there.

Criteria for identifying simple, transparent and comparable securitisations

(See BIS D304)

Our view is that securitisation is fundamental financial technology and there is no intrinsic technical reason why it could not be harnessed to best serve the functioning of modern economies.

We believe, though, that a comprehensive overhaul of historical securitisation practices is the best means of addressing the stigma that has been attached to it in the follow up to the recent financial crisis. The laudable objective of introducing criteria for simple, transparent and comparable securitisations, (STCS Criteria for short) should aim to achieve a quantum leap in transparency and avoid the risk of perceived ineffectual measures. Technically this would require a visible reduction of model risk for prospective investors, expressed for example in tangible new ability to perform relevant risk analyses.

Financial Risk Modelling has suffered enormous setbacks in recent years, with all major strands of modelling (market, credit, operational risk) proven to have debilitating limitations. It is impossible to imagine a modern financial system that does not make extensive use of risk quantification tools, yet rebuilding confidence that these tools are fit-for-purpose will require significant changes. These need to improve governance, transparency, quality standards and in some areas even the development of completely new strands of modelling.