Back to TILs.

Calmcode TIL

Matrix Cookbook logoMatrix Cookbook

Sometimes you're working with code that does maths and you just need that one formula, or proof, from linear algebra. It tends to pop up a lot in custom deep learning code, physical simulations or novel machine learning tricks. In these cases, try searching for "the matrix cookbook". It will take you to a hosted pdf that contains many of the formulas you might need.

The book contains proofs and formulas that involve:

  • derivatives
  • matrix inversion
  • matrices with complex numbers
  • decomposition proofs
  • probability distributions in higher dimensions

The cookbook is not meant for beginners. You really need to have taken a full course in linear algebra and probability theory to appreciate everything. But it's proven to be one of those books that can prevent a google-search.

An example

Let's say you appreciate a reminder on the conditional distribution of the Gaussian. Then on page 40, on section 8.1.3 you'll find:

The next page lists:


The matrix cookbook is mathematically dense. It certainly expects that you're mathematically experienced. But if you are, it's a nice reference to have. If you're interested you can download it from the university of Waterloo.

Back to TILs.