SVD

Singular value decomposition

For Singular value decomposition, people always call it SVD. And it is a way to break up any $m\times n$ (rectangular ) matrix into 3 pieces:

$A=U\Sigma V^{T}$

Where

$U$ is a $m\times m$ orthogonal matrix

$\Sigma$ is a $m\times n$ diagonal matrix, with singular value

$V^{T}$ is a $n\times n$ orthogonal matrix

Noted that we can rewrite any square matrix into:

$B=V\Lambda V^T$

Where $V$ and $V^T$ is the eigenvalue vector for matrix $B$ and $\Lambda $ is the diagonal matrix with eigenvalue from top-left to bottom-right as large value to small value.

What is the difference between this two methods:

Let’s see

$A^T A=(V\Sigma^T U^T)(U\Sigma V^{T})=V\Sigma^T \Sigma V^T$

Since $U$ is the orthogonal matrix, $U^TU=I$

And we can find the relation is that $A^T A$ is a square matrix and is positive definite since it is a square of a matrix $A$ . So the eigenvaues are greater or equal than 0. $V$ and $V^T$ is the eigenvalue vector for $A^TA$ and $\Sigma^T \Sigma$ is the $\lambda$ (eigenvalue) for $A^TA$ or the $\sigma ^2$ for the $A^TA$.

(Determinal is 0)= singular matrix, orthogonal matrix has determinant as 1, the product of $\Sigma$ is same as the determinant of $A$

Only for square matrix that $Ax=\lambda x$