Seeing the Shape of a Model

A geometric presentation of linear regression as a linear transformation from parameter space to data space.

As TA for a class called Theory of Predictive Modeling at BYU, I prepare weekly recitations. Much of the course content is about the geometry of modeling problems, and the four fundamental subspaces mapped onto regression problems. I’ve had a great time using Grant Sanderson’s incredible Manim package to create presentations for my recitations. I’ve linked to the .html slide decks, which need to be clicked through with the right arrow. You can also find the transcripts I wrote to accompany the slide decks. These aren’t quite finished form, but they worked for the first recitation!

In order, these slides 1) review bases, coordinates, and matrix multiplication, 2) review the four fundamental subspaces, 3) visualize a linear regression problem as a linear transformation, and 4) visualize the ways the four fundamental subspaces map onto linear regression problems (identifiability in parameter space, model manifold / model mismatch in data space).

1. Bases, coordinates, and matrix-vector multiplication — presentation | transcript
2. The four fundamental subspaces — presentation | transcript
3. Linear regression as a linear transformation — presentation | transcript
4. The four fundamental subspaces in regression — presentation | transcript