Scikit-learn Imputer Reducing Dimensions

Question

I have a dataframe with 332 columns. I want to impute values to be able to use scikit-learn's decision tree classifier. My problem is that the column of the resulting data from imputer function is only 330.

from sklearn.preprocessing import Imputer
imp = Imputer(missing_values='NaN', strategy='mean', axis=0)
cols = data.columns
new = imp.fit_transform(data)

print(data.shape,new.shape)
(34132, 332) (34132, 330)

Do you have all-NaN columns by any chance? – Ami Tavory Aug 11 '16 at 15:28 — Ami Tavory, Aug 11 '16 at 15:28
Yes. That's my answer. Thank you. – jrlund Aug 11 '16 at 15:52 — jrlund, Aug 11 '16 at 15:52

score 6 · Answer 1 · answered Aug 11 '16 at 19:03

6

According to the documentation of sklearn.preprocessing.Imputer:

When axis=0, columns which only contained missing values at fit are discarded upon transform.

So, this is removing all-missing-value columns.

answered Aug 11 '16 at 19:03

Ami Tavory

74,578
11
141
185

Scikit-learn Imputer Reducing Dimensions

1 Answers1

Linked