The code below doesn't drop all (it does drop some) columns where null values exist, how come? Does it make distinctions between None, np.nan, etc?
columns_to_drop = [x for x in X.columns if X[x].isnull().sum()>0]
print(columns_to_drop)
dropped_X = X.drop(columns_to_drop,axis = 1).copy()
Neither does the simpler:
X.dropna(axis=1, how='any')
Imputing values isn't an option.
Finally, I had to drop rows where missing values existed, after having dropped all columns where missing values existed (clearly I hadn't though)
columns_to_drop = [col for col in X2.columns if X[col].isnull().any()]
print(columns_to_drop)
dropped_X = X.drop(columns_to_drop,axis = 1).copy()
dropped_X2 = X2.drop(columns_to_drop,axis=1).copy()
dropped_X2.dropna(axis = 0, how = 'any',inplace = True)
Having ran that last line I was able to proceed. Why did I have to, though?
