Pandas fillna() not working on DataFrame slices

Question

Pandas fillna is not working on DataFrame slices, here is an example

df = pd.DataFrame([[np.nan, 2, np.nan, 0],
                [3, 4, np.nan, 1],
                [np.nan, np.nan, np.nan, 5],
                [np.nan, 3, np.nan, 4]],
                columns=list('ABCD'))
df[["A", 'B']].fillna(0, inplace=True)

the DataFrame doesn't change

    A   B   C   D
0   NaN 2.0 NaN 0
1   3.0 4.0 NaN 1
2   NaN NaN NaN 5
3   NaN 3.0 NaN 4

in contrast

df["A"].fillna(0, inplace=True)

and

df.fillna(0, inplace=True)

work fine.

Is this a bug or does it work as intended? Thx in advance.

P.S. this question asks how to use fillna on a slice, as for my question, it concerns why the above does'n work. The answer is in @heena-bawa answers comment section.

score 3 · Answer 1 · answered Apr 13 '19 at 11:28

3

If we look at the pandas documentation it says you should use the following to fillna on slices:

values = {'A':0, 'B':0}
df.fillna(value=values, inplace=True)

print(df)

     A    B   C  D
0  0.0  2.0 NaN  0
1  3.0  4.0 NaN  1
2  0.0  0.0 NaN  5
3  0.0  3.0 NaN  4

answered Apr 13 '19 at 11:28

Erfan

40,971
8
66
78

thanks for answering, but I think giving a `dict/Series` or `DF` is for "specifying which value to use for each index" and `df[["A", 'B']]` is a perfectly fine `DataFrame`. Your method is a good workaround but again my question was "Is this a bug or does it work as intended?". – HP6 Apr 13 '19 at 12:11

score 2 · Accepted Answer · answered Apr 13 '19 at 12:27

2

You can use:

df[['A','B']] = df[['A','B']].fillna(0)

     A    B   C  D
0  0.0  2.0 NaN  0
1  3.0  4.0 NaN  1
2  0.0  0.0 NaN  5
3  0.0  3.0 NaN  4

answered Apr 13 '19 at 12:27

heena bawa

818
6
5

1

isn't the purpose of `inplace=True` to avoid the use of assignment operator? – HP6 Apr 13 '19 at 12:42
inplace=True for slice of a dataframe won't work. It works on complete dataframe. You have to update the slice only. – heena bawa Apr 13 '19 at 12:46
For more information, you can refer: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html – heena bawa Apr 13 '19 at 12:50

score 0 · Answer 3 · answered Apr 13 '19 at 12:51

I think it works like intended. If I try your code it throws a warning:

SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  downcast=downcast, **kwargs)

Meaning: By selecting the columns in this way you're generating a copy and thus the parameter inplace can't have an effect.

Pandas fillna() not working on DataFrame slices

3 Answers3