4

When I multiply an eigenvector by a matrix, it should result in the same output as multiplying that eigenvector by its corresponding eigenvalue. I am trying to verify that my eigenvectors and eigenvalues are working as advertised, but the outputs don't seem right.

cov_matrix = np.cov(scaled_data)
eig_vals, eig_vecs = np.linalg.eigh(cov_matrix)

a = cov_matrix.dot(eig_vecs[:, 0])
b = eig_vecs[:, 0] * eig_vals[0]

When I print a and b, they are the same shape but their values are all different. What is going wrong here?

cmgodwin
  • 43
  • 4
  • 3
    the column eig_vecs[:, i] is the eigenvector corresponding to the eigenvalue eig_vals[i], not the row eig_vecs[i] – Yacola Jul 18 '20 at 17:33
  • Thank you! I edited the code (possibly incorrectly but not sure) to correct for this but a and b are still returning differing vectors. – cmgodwin Jul 18 '20 at 17:50
  • notice `np.linalg.eigh` works only on "complex Hermitian (conjugate symmetric) or a real symmetric matrix" (taken from documentation). Are you sure your matrix is symmetric? If so, please provide concrete example for `scaled_data` – Roim Jul 18 '20 at 17:58

1 Answers1

2

Try the following :

import numpy as np

np.random.seed(42) # for reproducibility
A = np.random.random((10,10)) + np.random.random((10,10)) * 1j
eig_vals, eig_vecs = np.linalg.eigh(A)

np.allclose(A @ eig_vecs[:, 0], eig_vals[0] * eig_vecs[:, 0])
>>> False

Keep in mind that np.linalg.eigh return the eigenvalues and eigenvectors of a complex Hermitian (conjugate symmetric) or a real symmetric matrix. So for a hermitian matrix:

A = (A + A.T.conj())/2  # Here A is forced Hermitian now
eig_vals, eig_vecs = np.linalg.eigh(A)

print(np.allclose(A @ eig_vecs[:, 0], eig_vals[0] * eig_vecs[:, 0]))
>>> True

Check prior to diagonalization if cov_matrix is symmetric with something like np.allclose(cov_matrix, cov_matrix.T.conj()). If not, you can just use np.linalg.eig.

Yacola
  • 2,873
  • 1
  • 10
  • 27
  • `cov_matrix` is indeed symmetric, I checked with your suggestion of using `np.allclose(cov_matrix, cov_matrix.T)`. Also, when substituting `A = np.random.random((10,10))` with `A=cov_matrix`, your code returned true. I think there's something wrong with the way I'm checking the values. – cmgodwin Jul 18 '20 at 18:17
  • I was doing `print(a[0])`, `print(b[0])` and `print(a[0] == b[0])`, with the idea that the values would be identical, but this must be misguided somehow. – cmgodwin Jul 18 '20 at 18:21
  • Okay, I figured out the problem, which actually related to your initial comment of the columns being the eigenvectors. I was using eig_vecs = eig_vecs[::-1] to flip the list order, which was jumbling all the eigenvectors. `np.allclose` then confused me for still returning true in some cases, but that was just for negligible eigenvalues so the vectors looked close enough. Thanks so much for your help, I would've stared at this for several more hours. – cmgodwin Jul 18 '20 at 19:25