#vectors #linearalgebra #matrices
The Singular Value Decomposition is one of the most important algorithms in linear algebra. It looks for the ellipse that is hidden in all linear transformations. The ellipse reveals the most important "directions" of the transformation, so that we can extract the most meaningful concepts from a huge data set. We show how this works for the classification of human faces and for movie review prediction. Welcome to the world of artificial intelligence.
Small correction at 21:12: Some of the columns do have non-zero coordinates in common of course, but they happen to cancel each other out, so the dot product is still zero.
To watch exclusive content, and to get access to new videos many months before they appear on KZread, consider supporting us on Patreon: www.patreon.com/user?u=86649007
If you want to dig deeper into the singular value decomposition, here's a long list of interesting links for you to explore:
[SERR 1] • Singular Value Decompo...
One of the best introductions to the SVD. It clearly shows the rotation, scaling, and second rotation. Also shows how we can compress a matrix by ignoring small singular values. And then he applies the SVD to image compression, by removing the least impactful singular values. Very cool!
[SERR 2] • Principal Component An...
Principal component analysis.
[SERR 3] • The covariance matrix
The covariance matrix.
[STO 1] • What is the Singular V...
Explains the SVD from a different angle. Instead of trying to map a circle to an ellipse, we ask: how do we maintain orthogonality? The result is again the SVD.
[AIO 1] • Lecture 47 - Singular ...
[AIO 2] • Lecture 50 - SVD Examp...
Using the SVD to discover movie genres from review data, and to predict whether or not a specific user will like a specific movie.
[AIO 3] • Lecture 48 - Dimension...
These SVD techniques allow you to compress your data to much smaller sizes, and to reduce its number of dimensions to make it easier to analyze.
[SB 1] • Singular Value Decompo...
This is a very good series about the SVD. It's explained very clearly, and it shows many concrete examples with actual Python code.
[SB 2] • SVD: Image Compression...
Steve Brunton shows actual Python code for compressing an image of his dog. He also plots the size of the singular values so that you can gauge how many of them are small enough to ignore.
[MIT 1] • 25. Symmetric Matrices...
Proves that a symmetric matrix always has real eigenvalues and orthogonal eigenvectors.
[BN 1] • Matrix Transpose and t...
This video is a bit more theoretical and deep, but it draws some interesting connections. It uses the SVD to look at 4 fundamental spaces for a matrix, which gives us an alternative intuition about the [/transpose].
0:00 Introduction
2:59 Decomposing a matrix into 3 parts
5:44 The covariance matrix
11:55 PCA in higher dimensions
13:55 Human faces
17:59 Movie reviews
23:25 How to calculate the SVD
28:39 Please support our channel
29:04 Comparison between eigenstuff and SVD
This video is published under a CC Attribution license
( creativecommons.org/licenses/... )