15

I'm trying to do a PCA analysis on a masked array. From what I can tell, matplotlib.mlab.PCA doesn't work if the original 2D matrix has missing values. Does anyone have recommendations for doing a PCA with missing values in Python?

Thanks.

Emily
  • 825
  • 3
  • 10
  • 20
  • 1
    You might look up non-linear iterative partial least squares (http://en.wikipedia.org/wiki/Non-linear_iterative_partial_least_squares). I don't know whether there are any Python implementations, but the algorithm works fine with missing values, so if you can find an implementation (or write it yourself!) you should be able to it. – DavidW Apr 02 '15 at 22:27

2 Answers2

19

Imputing data will skew the result in ways that might bias the PCA estimates. A better approach is to use a PPCA algorithm, which gives the same result as PCA, but in some implementations can deal with missing data more robustly.

I have found two libraries. You have

  1. Package PPCA on PyPI, which is called PCA-magic on github
  2. Package PyPPCA, having the same name on PyPI and github

Since the packages are in low maintenance, you might want to implement it yourself instead. The code above build on theory presented in the well quoted (and well written!) paper by Tipping and Bishop 1999. It is available on Tippings home page if you want guidance on how to implement PPCA properly.

As an aside, the sklearn implementation of PCA is actually a PPCA implementation based on TippingBishop1999, but they have not chosen to implement it in such a way that it handles missing values.

EDIT: both the libraries above had issues so I could not use them directly myself. I forked PyPPCA and bug fixed it. Available on github.

LudvigH
  • 3,662
  • 5
  • 31
  • 49
  • 1
    For those looking to compute PC coordinates for incoming data after performing the decomposition with PyPPCA, the answer is in equation 12 of the [publication](https://hal.inria.fr/inria-00321476/en). y = (ss*np.eye(size) + C_o@C_o.T)@C_o@z_o. Where z is the new data with missing values and the _o refers to only the "observed" rows. Now I just need to figure out how to handle the error that these matrices are too big in my case. – kjohnsen Dec 19 '20 at 13:25
  • Whoops, forgot the np.linalg.inv at the beginning – kjohnsen Dec 19 '20 at 13:51
  • 1
    Careful: output of PyPPCA has flipped dimensions from that of the paper (e.g., in the paper C is dxD but the output of the program is Dxd) – kjohnsen Dec 19 '20 at 14:38
11

I think you will probably need to do some preprocessing of the data before doing PCA. You can use:

sklearn.impute.SimpleImputer

https://scikit-learn.org/stable/modules/generated/sklearn.impute.SimpleImputer.html#sklearn.impute.SimpleImputer

With this function you can automatically replace the missing values for the mean, median or most frequent value. Which of this options is the best is hard to tell, it depends on many factors such as how the data looks like.

By the way, you can also use PCA using the same library with:

sklearn.decomposition.PCA

http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html

And many others statistical functions and machine learning tecniques.

javiercmh
  • 9
  • 3
Numlet
  • 819
  • 1
  • 9
  • 21