Open Risk Academy Course: Tensor Calculations with the Eigen C++ Library

Course Objective

The objective of the course is to provide an introduction to using Eigen::Tensor as a high-level library for using Tensors in C++ projects.

• We learn the concept and techniques of the Eigen Tensor class
• How to declare, initialize Tensors of various ranks and types and how to access Tensor elements
• Elementary unary and binary operations involving Tensors
• More complex operations (reductions, contractions)
• Modifying the shape of Tensors

The course is now live at the Academy, the github repository hosts C++ scripts used in the course.

What is Eigen and what does it look like?

Eigen is an open source C++ template library for linear algebra: It provides containers for vectors, matrices, tensors, alongside with various methods, operations, numerical solvers, and related algorithms that allow the simple and efficient calculation. The name Eigen is a play on eigenvalues (or eigenvectors) which is a frequent issue in numerical linear algebra.

Eigen is a header-only library. This means you simply drop the source code in some directory of your project, include the relevant headers in the right places, and you are ready to get going. Here is a preview of what a calculation with Eigen might look like:

    Eigen::Tensor<int, 2> A(2, 3);
    Eigen::Tensor<int, 2> B(3, 2);
    A.setValues({{1, 1, 1}, {1, 1, 1}});
    B.setValues({{1, 1}, {2, 2}, {3, 3}});
    Eigen::array<Eigen::IndexPair<int>, 1> dims = { Eigen::IndexPair<int>(1, 0) };
    Eigen::Tensor<int, 2> AB = A.contract(B, dims);

Where to Get it

• Code development happens on gitlab from where you can get a copy
• From the conan C/C++ package manager (as a recipe)
• Documentation is also hosted online by tuxfamily
• Chat like discussion on discord
• Mailing list

The Structure of the Tensor library

The Tensor part of Eigen lives under the unsupported/Eigen/CXX11 directory. The “unsupported” part is somewhat of a misnomer with historical roots.

There are currently three components of the Tensor library:

• The Tensor class (the workhorse class and the only one we mostly focus on)
• The TensorSymmetry class (for tensors that obey certain symmetries between different dimensions)
• The ThreadPool class providing ability to compute expressions in native threads managed by the Eigen library

The course focuses on the first class (Tensor) that supports the declaration and numerical manipulation of multi-dimension tensor objects.

What is Eigen good for?

Eigen is useful for developing applications and tools that process significant amounts of multidimensional numerical data. A very common domain is numerical linear algebra applications. The role of numerical computation libraries and frameworks such as Eigen includes (at least) the following:

• Enable an efficient way of storing and accessing, reshaping etc. large numbers of numbers in memory.
• Enable basic numerical operations on such collections of numbers (arithmetic operations, function evaluation and more).
• Enable more sophisticated (but still standard or known) algorithms (linear algebra etc.) operating on these collections.
• Enable the performant implementation of novel algorithms.

The design focus of numerical libraries is on striking a balance of performance versus an easy-to-use interface (API). In this course we focus on higher-dimensional tensors. For vectors and matrices Eigen offers substantially more functionality (e.g. ability to work with Sparce matrices).

Pre-requisites

Basic knowledge and a working setup for C++ development (e.g., being able to add Eigen as a header only library) is required. Mathematical notation is used liberally throughout the course to clarify (for those familiar with it) the tensor manipulation concepts but is not strictly required for benefiting from the course.

Summary of the Course

What we cover in this course.

• Step 1. Getting started with Eigen
• Step 2. Tensor Class Declarations
• Step 3. Tensor Class Initializations
• Step 4. Working with Tensor Elements, I
• Step 5. Working with Tensor Elements, II
• Step 6. Random Number Initialization
• Step 7. Unary Element-Wise Operations
• Step 8. Binary Element-Wise Operations
• Step 9. In-Memory Representations of Tensors
• Step 10. Tensor Contraction Operations
• Step 11. Tensor Reduction Operations
• Step 12. Tensor Shape Modifying Operations
• Step 13. Tensor Scanning Operations

Get Started with the Course

The course is available at the Academy, while the github repository hosts C++ scripts used in the course.