fillna does fill the dataframe in the NaN cells

Question

What am I missing? fillna doesn't fill NaN values:

#filling multi columns df with values..

df.fillna(method='ffill', inplace=True)
df.fillna(method='bfill', inplace=True)

#just for kicks
df = df.fillna(method='ffill')
df = df.fillna(method='bfill')

#retun true
print df.isnull().values.any()

I verified it - I actually see NaN values in some first cells..

Edit So I'm trying to write it myself:

def bfill(df):
    for column in df:
        for cell in df[column]:
            if cell is not None:
                tmpValue = cell
                break

        for cell in df[column]:
            if cell is not None:
                break
            cell = tmpValue

However it doesn't work... Isn't the cell is by ref?

score 0 · Answer 1 · answered Dec 06 '17 at 22:21

0

ffill fills rows with values from the previous row if they weren't NaN, bfill fills rows with the values from the NEXT row if they weren't NaN. In both cases, if you have NaNs on the first and/or last row, they won't get filled. Try doing both one after the other. If any columns have entirely NaN values then you will need to fill again with axis=1, (although I get a NotImplementedError when I try to do this with inplace=True on python 3.6, which is super annoying, pandas!).

answered Dec 06 '17 at 22:21

Turksarama

1,136
6
13

Each column has data in the middle. But the begging and maybe some in the end are the problem. what do you mean do both one after the other? this is what i'm doing.. no? – Nir Dec 07 '17 at 17:35

score 0 · Accepted Answer · answered Dec 10 '17 at 22:26

So, I don't know why but taking the fillna outside the function fixed it..

Origen:

def doWork(df):
  ...
  df = df.fillna(method='ffill')
  df = df.fillna(method='bfill')

def main():
  ..
  doWork(df)
  print df.head(5) #shows NaN

Solution:

def doWork(df):
  ...

def main():
  ..
  doWork(df)

  df = df.fillna(method='ffill')
  df = df.fillna(method='bfill')

  print df.head(5) #no NaN

fillna does fill the dataframe in the NaN cells

2 Answers2