Subset columns with NaN values in Pandas

Question

Searched and tried several answers here on SO, but they are all for returning rows with NaN's. I'd like to return only the columns with NaN values. For example the following df. How can I select columns 'A' and 'LG'?

df = pd.DataFrame(
        {'H': ['a','b', 'c'],
         'A': [np.nan,'d', 'e'],
         'LG':['AR1', 'RO1', np.nan],
         })

print(df)

     A  H   LG
0  NaN  a  AR1
1    d  b  RO1
2    e  c  NaN

possible dupe: [How to find which columns contain any NaN value in Pandas dataframe (python)](http://stackoverflow.com/questions/36226083/how-to-find-which-columns-contain-any-nan-value-in-pandas-dataframe-python) — MaxU - stand with Ukraine, Apr 27 '17 at 08:16
in part I believe so. I'm asking for a subset, however that one is returning the column names only — Zanshin, Apr 27 '17 at 08:30
good point, thanks! I've updated [that post](http://stackoverflow.com/a/36226137/5741205) — MaxU - stand with Ukraine, Apr 27 '17 at 08:37

jezrael · Accepted Answer · 2017-04-27T08:13:39.463

I think you need first replace strings NaN to np.nan in sample:

df = pd.DataFrame(
        {'H': ['a','b', 'c'],
         'A': [np.nan,'d', 'e'],
         'LG':['AR1', 'RO1', np.nan],
         })

Then check by isnull and any:

mask = df.isnull().any()
print (mask)
A      True
H     False
LG     True
dtype: bool

Last use boolean indexing with index:

print (mask.index[mask])
Index(['A', 'LG'], dtype='object')

If need columns add loc:

print (df.loc[:, mask])
     A   LG
0  NaN  AR1
1    d  RO1
2    e  NaN

score 1 · Answer 2 · answered Apr 27 '17 at 08:11

This returns all columns containing NaN.

df = pd.DataFrame(
        {'H': ['a','b', 'c'],
         'A': [np.nan,'d', 'e'],
         'LG':['AR1', 'RO1', np.nan],
         })

x = pd.isnull(df).any()

print(df[x.index[x]])

     A   LG
0  NaN  AR1
1    d  RO1
2    e  Na

Subset columns with NaN values in Pandas

2 Answers2