keyError when trying to drop a column in pandas.

Question

I want to drop some rows from the data. I am using following code-

    import pandas as pd
    import numpy as np

    vle = pd.read_csv('/home/user/Documents/MOOC dataset original/vle.csv')


    df = pd.DataFrame(vle)
    df.dropna(subset = ['week_from'],axis=1,inplace = True)
    df.dropna(subset = ['week_to'],axis=1,inplace = True)
    df.to_csv('/home/user/Documents/MOOC dataset cleaned/studentRegistration.csv')

but its throwing following error-

      raise KeyError(list(np.compress(check,subset)))
      KeyError: [' week_from ']

what is going wrong?

I am a bit confused - need remove columns by names or rows by `NaN`s specifying by columns? — jezrael, Jul 08 '18 at 13:48
question has nothing to do with `machine-learning` - kindly do not spam the tag (removed) — desertnaut, Jul 08 '18 at 14:05

jezrael · Accepted Answer · 2018-07-08T13:49:22.010

I think need omit axis=1, because default value is axis=0 for remove rows with NaNs (missing values) by dropna by subset of columns for check NaNs, also solution should be simplify to one line:

df.dropna(subset = ['week_from', 'week_to'], inplace = True)

Sample:

df = pd.DataFrame({'A':list('abcdef'),
                   'week_from':[np.nan,5,4,5,5,4],
                   'week_to':[1,3,np.nan,7,1,0],
                   'E':[5,3,6,9,2,np.nan],
                   'F':list('aaabbb')})

print (df)
   A  week_from  week_to    E  F
0  a        NaN      1.0  5.0  a
1  b        5.0      3.0  3.0  a
2  c        4.0      NaN  6.0  a
3  d        5.0      7.0  9.0  b
4  e        5.0      1.0  2.0  b
5  f        4.0      0.0  NaN  b

df.dropna(subset = ['week_from', 'week_to'], inplace = True)
print (df)
   A  week_from  week_to    E  F
1  b        5.0      3.0  3.0  a
3  d        5.0      7.0  9.0  b
4  e        5.0      1.0  2.0  b
5  f        4.0      0.0  NaN  b

If want remove columns by specifying rows for check NaNs:

df.dropna(subset = [2, 5], axis=1, inplace = True)
print (df)
   A  week_from  F
0  a        NaN  a
1  b        5.0  a
2  c        4.0  a
3  d        5.0  b
4  e        5.0  b
5  f        4.0  b

But if need remove columns by names solution is different, need drop:

df.drop(['A','week_from'],axis=1, inplace = True)
print (df)
   week_to    E  F
0      1.0  5.0  a
1      3.0  3.0  a
2      NaN  6.0  a
3      7.0  9.0  b
4      1.0  2.0  b
5      0.0  NaN  b

keyError when trying to drop a column in pandas.

1 Answers1

Linked