61

If I have a dataframe with multiple columns ['x', 'y', 'z'], how do I forward fill only one column 'x'? Or a group of columns ['x','y']?

I only know how to do it by axis.

Shaido
  • 27,497
  • 23
  • 70
  • 73
azuric
  • 2,679
  • 7
  • 29
  • 44

6 Answers6

113

tl;dr:

cols = ['X', 'Y']
df.loc[:,cols] = df.loc[:,cols].ffill()

And I have also added a self containing example: 

>>> import pandas as pd
>>> import numpy as np
>>> 
>>> ## create dataframe
... ts1 = [0, 1, np.nan, np.nan, np.nan, np.nan]
>>> ts2 = [0, 2, np.nan, 3, np.nan, np.nan]
>>> d =  {'X': ts1, 'Y': ts2, 'Z': ts2}
>>> df = pd.DataFrame(data=d)
>>> print(df.head())
    X   Y   Z
0   0   0   0
1   1   2   2
2 NaN NaN NaN
3 NaN   3   3
4 NaN NaN NaN
>>> 
>>> ## apply forward fill
... cols = ['X', 'Y']
>>> df.loc[:,cols] = df.loc[:,cols].ffill()
>>> print(df.head())
   X  Y   Z
0  0  0   0
1  1  2   2
2  1  2 NaN
3  1  3   3
4  1  3 NaN
Hennep
  • 1,602
  • 1
  • 11
  • 20
  • This throws up a `SettingWithCopyWarning`. Any thoughts on worrying about that warning? – yeliabsalohcin May 14 '19 at 11:08
  • 2
    See the description in here about the preferred way of indexing in Python: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html Changing to .loc statements should solve the issue, I have updated the answer accordingly. – Hennep Jun 12 '19 at 11:47
  • If I hope to exclude columns X and Y when I fillna? – ah bon Dec 02 '21 at 15:14
  • ```## apply forward fill with exclusion cols_ex = ['X', 'Y'] # define columns to exclude \\ cols = df.columns # get list with all columns \\ cols_to_ff = cols.drop(cols_ex) # drop columns to exclude \\ df.loc[:,cols_to_ff] = df.loc[:,cols_to_ff].ffill() # forward fill the selected columns \\ ``` – Hennep Dec 24 '21 at 16:45
19
for col in ['X', 'Y']:
    df[col] = df[col].ffill()
Woody Pride
  • 13,539
  • 9
  • 48
  • 62
11

Alternatively with the inplace parameter:

df['X'].ffill(inplace=True)
df['Y'].ffill(inplace=True)

And no, you cannot do df[['X','Y]].ffill(inplace=True) as this first creates a slice through the column selection and hence inplace forward fill would create a SettingWithCopyWarning. Of course if you have a list of columns you can do this in a loop:

for col in ['X', 'Y']:
    df[col].ffill(inplace=True)

The point of using inplace is that it avoids copying the column.

Uwe Mayer
  • 736
  • 7
  • 12
4

I used below code, Here for X and Y method can be different also instead of ffill().

 df1 = df.fillna({
        'X' : df['X'].ffill(),
        'Y' : df['Y'].ffill(),
    })
Abhishek Chaurasia
  • 816
  • 6
  • 4
4

Two columns can be ffill() simultaneously as given below:

df1 = df[['X','Y']].ffill()
Amit Joshi
  • 15,448
  • 21
  • 77
  • 141
Souvik Daw
  • 129
  • 7
  • 1
    I tried this solution first as it seems cleaner to me but one problem with this solution is that only columns X and Y are saved in the df1 dataframe. The remaining column(s) are not. Both Woody Pride's and Abhishek Chaurasia responses both retain all columns. – burkesquires Jun 11 '21 at 21:00
1

The simplest version I think.

cols = ['X', 'Y']
df[cols] = df[cols].ffill()
Bernard Agbemadon
  • 101
  • 5