Data Types are a fundamental building block of data science Data science is about data, but data are not simple and tame beasts. They have character and attitude, which can cause a lot of friction between them and the data scientist. There is a lot of sweat and tears involved when confronting data, but data scientists can do worse than know how to handle in particular Data Type quirks. Namaly a good fraction of data science involves not modelling data, not transforming data, not even cleaning data but simply goading data around the right containers, providing them with the right stage that fits their character.
Course Content: This CrashCourse is an introduction to semantic data using Python. It covers the following topics: We learn to work with RDF graphs using rdflib We explore the owlready package and OWL ontologies We look into json-ld serialization of RDF/OWL data We try data validation using pySHACL We use throughout a realistic data set based on the Credit Ratings Ontology Who Is This Course For: The course is useful to:
Introduction: What is FOSDEM? FOSDEM is a non-commercial, volunteer-organized event centered on free and open-source software development (with a geographic focus on the European open source ecosystems / projects). FOSDEM is aimed at developers and anyone interested in the free and open-source software movement. It aims to enable developers to meet and to promote the awareness and use of free and open-source software. FOSDEM is held annually since 2001, usually during the first weekend of February, at the Université Libre de Bruxelles Solbosch campus in the southeast of Brussels, Belgium.
Using Sankey Diagrams: Sankey Diagrams are a type of flow diagram composed of interconnected arrows. The width of the arrows is proportional to the flow rate. Sankey diagrams are often used in physical sciences (physics, chemistry, biology) and engineering but also in economics. They can be used to represent the relative role and significance of various inputs and outputs in a given process. Sankey diagrams emphasize the major transfers within a system.
Risk Management will not be the same going forward: too much is at stake! The summer is over in the Northern Hemisphere - and what an unusual summer has it been! Worldwide the implications and challenges of adjusting to a Covid-19 pandemic are still a major issue, affecting individuals, companies and governments. At Open Risk we have been tracking and will continue to interpret the impact of the pandemic via a number of projects:
What this blog post is about (and what it isn’t): With the ever more widespread adoption of Data Science, defined as the intensive use of data in various forms of decision making, there is a renewed interest in Visualization as an effective channel for humans to understand data at various stages of the data lifecycle. There is a large variety of data visualization tools which can produce an ever more bewildering variety of visualization types
Non-Performing Loans: The covid-19 crisis will certainly impact the concentration of Non-Performing Loans but given the special nature of this economic crisis compared (in particular) with the 2008 financial crisis it is unclear how precisely things will evolve. In a previous post and white paper (OpenRiskWP07_022616) we discussed the importance of advancing open and transparent methodologies for managing the risks associated with such credit portfolios. Effective management of NPL is also a top regulatory priority.
Course Content: This course is a CrashProgram (short course) introducing the GeoJSON specification for the encoding of geospatial features. The course is at an introductory technical level. It requires some familiarity with data specifications such as JSON and a very basic knowledge of Python Who Is This Course For: The course is useful to: Any developer or data scientist that wants to work with geospatial features encoded in the geojson format How Does The Course Help: Mastering the course content provides background knowledge towards the following activities:
Introduction: FOSDEM is a non-commercial, volunteer-organized European event centered on free and open-source software development. It is aimed at developers and anyone interested in the free and open-source software movement. It aims to enable developers to meet and to promote the awareness and use of free and open-source software. FOSDEM is held annually since 2001, usually during the first weekend of February, at the Université Libre de Bruxelles Solbosch campus in the southeast of Brussels, Belgium.
The motivation for federated credit risk models: Federated learning is a machine learning technique that is receiving increased attention in diverse data driven application domains that have data privacy concerns. The essence of the concept is to train algorithms across decentralized servers, each holding their own local data samples, hence without the need to exchange potentially sensitive information. The construction of a common model is achieved through the exchange of derived data (gradients, parameters, weights etc).
Overview of the Julia-Python-R Universe: A new Open Risk Manual entry offers a side-by-side review of the main open source ecosystems supporting the Data Science domain: Julia, Python, R, sometimes abbreviated as Jupyter. Motivation A large component of Quantitative Risk Management relies on data processing and quantitative tools (aka Data Science ). In recent years open source software targeting Data Science finds increased adoption in diverse applications. The overview of the Julia-Python-R Universe article is a side by side comparison of a wide range of aspects of Python, Julia and R language ecosystems.
Open Source Securitisation: Motivation After the Great Financial Crisis securitisation has become the poster child of a financial product exhibiting complexity and opaqueness. The issues and lessons learned post-crisis were many, involving all aspects of the securitisation process, from the nature and quality of the underlying assets, the incentives of the various agents involved and the ability of investors to analyze the products they invested in. While the most egregious complications involved various types of re-securitisation and/or the interplay of structured credit derivatives undoubtedly even vanilla securitisation structure has a considerable amount of business logic.
Motivation for the comparison: A large component of risk management relies on data processing and quantitative tools. In turn, such information processing pipelines and numerical algorithms must be implemented in computer systems. Computing systems come in an extraordinary large variety but in recent years open source software finds increased adoption for diverse applications (machine learning, data science, artificial intelligence). In particular cloud computing environments are primarily based on open source projects at the systems level.
Release of version 0.4 of the Concentration Library adds Geographic / Industrial concentration indexes: Further building out the OpenCPM set of tools, we release version 0.4 of the Concentration Library, a python library for the computation of various concentration, diversification and inequality indices. The below list provides documentation URL’s for each one of the implemented classic indexes (the Hoover index is a new addition in this release Atkinson Index Hoover Index Concentration Ratio Berger-Parker Index Herfindahl-Hirschman Index Hannah-Kay Index Gini Index Theil Index Shannon Index Generalized Entropy Index Kolm Index An important new direction that appears first in this release is the introduction of indexes that measure geographical and industrial concentration.
Release of version 0.3 of the Concentration Library: Further building out the OpenCPM set of tools, we release version 0.3 of the Concentration Library. This python library for the computation of various concentration, diversification and inequality indices. The below list provides documentation URL’s for each one of the implemented indexes Atkinson Index Concentration Ratio Berger-Parker Index Herfindahl-Hirschman Index Hannah-Kay Index Gini Index Theil Index Shannon Index Generalized Entropy Index Kolm Index The image illustrates a simple use of the library where the HHI and Gini indexes are computed and compared for a range of randomly generated portfolio exposures.
Open Risk released version 0.1 of the Transition Matrix Library Motivation: State transition phenomena where a system exhibits stochastic (random) migration between well defined discrete states (see picture below for an illustration) are very common in a variety of fields. Depending on the precise specification and modelling assumptions they may go under the name of multi-state models, Markov chain models or state-space models. In financial applications a prominent example of phenomena that can be modelled using state transitions are credit rating migrations of pools of borrowers.
Loan Level Templates Using Python: In this Open Risk Academy course we figure step by step how to use python to work with Loan Level Templates, using the ECB SME template as an example. Overview of the loan level template Manipulating spreadsheets with Python The Python Dictionary Organization of Portfolio Data Generating Test Portfolios Get an Open Risk Academy account and get started with the course here
How much digital bank can we fit in a 50 euro bill? Much has been said about the impact of Big Data and high-end GPU computing on the provision of digital financial services. At Open Risk we wanted to explore the boundary of what is possible at the diametrically opposite end of the cost spectrum: What is the absolutely minimum cost for providing digital financial services? . In this post we begin the journey of finding out the answer to that question and it promises to be fascinating!
Open Source Risk Data with MongoDB and Python: Open source software is all the rage those days in IT and the concept is making rapid inroads in all parts of the enterprise. An earlier comprehensive survey by Gartner, Inc. found that by 2011 more than half of organizations surveyed had adopted open-source software (OSS) solutions as part of their IT strategy. This percentage may have currently exceeded the 75% mark according to open source advisory firms.
Open Source Risk Modeling Manifesto: This post is a summary of a presentation given at the 2014 Autumn TopQuants Meeting, aka, the Open Source Risk Modeling Manifesto. The dismal state of quantitative risk modeling The current framework of internal risk modeling at financial institutions has had a fatal triple stroke. We saw in quick sequence: market risk, operational risk, and credit risk measurement failures, covering practically all business models. This fact left the science and art of quantitative risk modeling reeling under the crushing weight of empirical evidence.