I'm implementing an SVD algorithm and wanting to use the .eig method (following this tutorial here: https://datajobs.com/data-science-repo/SVD-Tutorial-[Kirk-Baker].pdf)
We are given a 2x3 matrix:
A = [[3,1,1],[-1,3,1]]
And the first step is to work out the eigenvalues and eigenvectors for AA.T.
np.linalg.eig correctly returns [12, 10] as the eigenvalues.
The eigenvectors, however, for eigenvalue 10 are returned in the opposite order to that in the tutorial. That is, the tutorial says that for this eigenvalue, the eigenvector is [1, -1]. Numpy would say the unnormalised eigenvector is [-1, 1]. Both are valid solutions.
However, using Numpy's [-1, 1] leads to the reconstruction of the matrix with the rows permuted:
[[-1,3,1],[3,1,1]].
If I use the SVD method however, I am returned the correct matrix.
Code:
A = np.array([[3,1,1],[-1,3,1]])
## Find U (i.e. calculate AA^T)
U_ = np.matmul(A, A.T)
print(U_)
## Calculate the eigenvectors/eigenvalues of U
U_eig_values, U_eig_vectors = np.linalg.eig(U_)
print(U_eig_values)
# [12., 10.]
print(U_eig_vectors)
# [[ 0.70710678 -0.70710678]
# [ 0.70710678 0.70710678]]
## Find V (i.e. calculate A^TA)
V_ = np.matmul(A.T, A)
## Calculate the eigenvectors/eigenvalues of V
V_eig_values, V_eig_vectors = np.linalg.eig(V_)
print(V_eig_values)
# [ 0., 10., 12.]
## Transpose and sort V
V_eig_vectors = -1 * V_eig_vectors[:, [2,1,0]].T
print(V_eig_vectors)
# [[ 4.08248290e-01, 8.16496581e-01, 4.08248290e-
# [ 8.94427191e-01, -4.47213595e-01, -5.07702013e-
# [-1.82574186e-01, -3.65148372e-01, 9.12870929e-01]]
## Initialise S as an m x n matrix
m = len(U[0])
n = len(V[0])
S = np.zeros((m, n))
## Populate the diagonal of S with the square root of the non-zero eigenvalues from U and V.
## The diagonal should be in decending order (from largest to smallest) of these values.
np.fill_diagonal(S, np.sqrt(U_eig_values))
## Apply the SVD formula: A = USV.T
first_part = U_eig_vectors.dot(S)
reconstructed_A = first_part.dot(V_eig_vectors)
print(reconstructed_A)
# [[-1. 3. 1.]
# [ 3. 1. 1.]]
As mentioned, using the SVD method returns a correct reconstruction
u, s, vh = np.linalg.svd(A)
print(u)
print(s)
print(vh)
## Initialise S as an m x n matrix
m = len(u)
n = len(vh)
S = np.zeros((m, n))
## Populate the diagonal of S with the square root of the non-zero eigenvalues from U and V.
## The diagonal should be in decending order (from largest to smallest) of these values.
np.fill_diagonal(S, np.sqrt(U_eig_values))
## Apply the SVD formula: A = USV.T
first_part = u.dot(S)
reconstructed_A = first_part.dot(vh)
print(reconstructed_A)
# [[ 3. 1. 1.]
# [-1. 3. 1.]]
How can I use the .eig method to return the reconstructed matrix in the correct order, WITHOUT having to do something hacky like U_eig_vectors = U_eig_vectors[[1,0],:]?
