Representing a Sparse Matrix as a JSON object is a task that appears in many modern data science contexts. While there is no universally agreed way to achieve this task, in this post we discuss a number of options and the associated tradeoffs.
Recap of Part 1 of the Matrix-to-JSON Post Series In the first installment of this series, Part 1 we discussed the motivation behind representing and serializing matrices as JSON objects. We defined relevant concepts and in particular the concept of unrolling the matrix into a one-dimensional array and the notion of Column and Row Major orders. We outlined some use cases of interest and initiated a benchmarking exercise that looks into various R and Python JSON serialization utilities (available at the matrix2json repository).
What do we mean by credit data? This post is a discussion around mathematical terminology and concepts that are useful in the context of working with credit data, taking us from network graph representations of credit systems to commonly used reference data sets
Course Objective Digging into the meaning of credit data collections, the logic that binds them together towards understanding what they can be used for and what limitations and issues they may be affected by, this new course in the Credit Portfolio Management category explores a new angle to look at an old practice.
The course is now live at the Academy.
Pre-requisites Familiarity with credit provision in general (lending products, banking processes and credit risk) is required for getting the most out of the course.
Summary This course is a CrashProgram (short course) in the use of Python and the package TransitionMatrix for analysing credit migration data.
Requirements The course is at a medium technical level. It requires some familiarity with python (and a working installation that includes the common numpy/scipy libraries). On the risk modelling side it requires knowledge of basic credit rating migration concepts.
Outcomes Step by step we build the knowledge required to use python to analyse credit migration data:
Representing a matrix as a JSON object is a task that appears in many modern data science contexts, in particular when one wants to exchange matrix data online. While there is no universally agreed way to achieve this task in all circumstances, in this series of posts we discuss a number of options and the associated tradeoffs.
Motivation and Objective Representing a Matrix as a JSON object is a task that appears in many modern data science contexts, in particular when one wants to exchange matrix data online in a portable manner. There is no universally agreed way to achieve this task and various options are available depending on the matrix data characteristics and the programming tools and computational environment one has available.
Matrices are not, in general, native structures in general purpose computing environments.
Sankey diagrams are very useful for the visualization of flows, especially when there is a conserved quantity. They can be tricky when some of the flows are much smaller than others. In the latest release of transitionMatrix we include an example of a log-scale version of Sankey
Using Sankey Diagrams Sankey Diagrams are a type of flow diagram composed of interconnected arrows. The width of the arrows is proportional to the flow rate. Sankey diagrams are often used in physical sciences (physics, chemistry, biology) and engineering but also in economics. They can be used to represent the relative role and significance of various inputs and outputs in a given process.
Sankey diagrams emphasize the major transfers within a system.
Release of version 0.4.1 of the transitionMatrix package focuses on stressing transition matrices Further building the open source OpenCPM toolkit this realease of transitionMatrix features:
Feature: Added functionality for conditioning multi-period transition matrices Training: Example calculation and visualization of conditional matrices Datasets: State space description and CGS mappings for top-6 credit rating agencies Conditional Transition Probabilities The calculation of conditional transition probabilities given an empirical transition matrix is a highly non-trivial task involving many modelling assumptions.
Release of version 0.4 of the transitionMatrix package Further building the open source OpenCPM toolkit this realease of transitionMatrix features:
Feature: Added Aalen-Johansen Duration Estimator Documentation: Major overhaul of documentation, now targeting ReadTheDocs distribution Training: Streamlining of all examples Installation: Pypi and wheel installation options Datasets: Synthetic Datasets in long format Enjoy!
Is the IFRS 9 or CECL standard more volatile? Its all relative Objective In this study we compare the volatility of reported profit-and-loss (PnL) for credit portfolios when those are measured (accounted for) following respectively the IFRS 9 and CECL accounting standards.
The objective is to assess the impact of a key methodological difference between the two standards, the so-called Staging approach of IFRS 9. There are further explicit differences in the two standards.
Credit Portfolio PnL volatility under IFRS 9 and CECL Objective We explore conceptually a selection of key structural drivers of profit-and-loss (PnL) volatility for credit portfolios when profitability is measured following the principles underpinning the new IFRS 9 / CECL standards
Methodology We setup stylized calculations for a credit portfolio with the following main parameters and assumptions:
A portfolio of 200 commercial loans of uniform size and credit quality Maturities extending from one to five annual periods A stylized transition matrix producing typical multiyear credit curves Correlation between assets typical for a single business sector and geography portfolio Focusing on PnL estimates one year forward, with PnL being impacted both by Realized Losses (defaults) and Provision variability (both positive and negative).
Transition Matrix Library First Release Open Risk released version 0.1 of the Transition Matrix Library
Motivation State transition phenomena where a system exhibits stochastic (random) migration between well-defined discrete states (see picture below for an illustration) are very common in a variety of fields. Depending on the precise specification and modelling assumptions they may go under the name of multi-state models, Markov chain models or state-space models.
In financial applications a prominent example of phenomena that can be modelled using state transitions are credit rating migrations of pools of borrowers.
