This is a quick demo of what Singular Value Decomposition does. The math is based on notes from Data Mining and Engineering.

## Math

Starting with data $$X$$ of size $$D \times N$$, which represents $$N$$ examples of $$D$$-dimensional data, if I run SVD on it and ask for $$K$$ dimensions, SVD will decompose it into

where $$S$$ is a diagonal matrix of size $$K \times K$$, $$U$$ and $$V$$ are orthonormal matrices, and if $$X$$ of size $$D \times N$$, then $$U$$ is size $$D \times K$$ and $$V$$ is of size $$K \times N$$.