Use this tag for pandas.DataFrame.fillna or pandas.Series.fillna
Fill missing values using the specified method for a DataFrame or a Series with pandas (see also interpolate, ffill, bfill)
Docs:
Questions tagged [fillna]
528 questions
2
votes
1 answer
Problem with NaN values and fillna function python pandas
I got an issue, hopefully someone has a great solution.
I am reading an Excel file. And I use keep_default_na=False because there is a productname called "NA" and I dont want pandas changing it to NaN.
df = pd.read_excel('Import_orders.xlsx',…
hii
- 77
- 6
2
votes
1 answer
fillna method is slower for square bracket column selection than loc for pandas dataframe
I found significant processing time difference in fillna for different column selection techniques of pandas dataframe.
Time taken for fillna of dataframe, whose columns are selected using loc
df1 = df.copy()
t1 = time.time()
df1.loc[:, col] =…
piyush-balwani
- 524
- 3
- 15
2
votes
1 answer
How to use fillna not to propagate last valid observation but to propagate last week valid observation
Here is an exctract of my dataset :
Dataset
Here is an exemple of dataset
df = pd.DataFrame(
{'vals': np.where(np.arange(35) < 30, np.arange(35), np.nan)},
index=pd.date_range('2021-01-01', freq='12H', periods=35))
…
Thomas LESIEUR
- 408
- 4
- 14
2
votes
2 answers
How to shift a dataframe element-wise to fill NaNs?
I have a DataFrame like this:
>>> df = pd.DataFrame({'a': list('ABCD'), 'b': ['E',np.nan,np.nan,'F']})
a b
0 A E
1 B NaN
2 C NaN
3 D F
I am trying to fill NaN with values of the previous column in the next row and dropping this…
imxitiz
- 3,920
- 3
- 9
- 33
2
votes
1 answer
Replace selected row values based on certain column
I would like to find rows in df where C contains X1 and replace value in A with value in C and, replace value in C with value in result1(shift values in C and result1 to the leftmost).
df:
A C result1 result2
0 NaN X1 …
nilsinelabore
- 4,143
- 17
- 65
- 122
2
votes
1 answer
df.fillna() not working when updating multiple columns from a slice
Having issues with fillna() and df slices. Still have my python training wheels on and would appreciate any assistance. I've found lots of close examples on SE, but because of the conditionals and multiple columns I haven't found anything that…
Donovin
- 55
- 7
2
votes
2 answers
How to use each vector entry to fill NAN's of a separate groups in a dataframe
Say I have a vector ValsHR which looks like this:
valsHR=[78.8, 82.3, 91.0]
And I have a dataframe MainData
Age Patient HR
21 1 NaN
21 1 NaN
21 1 NaN
30 2 NaN
30 2 NaN
24 3 NaN
24 …
sahmed
- 69
- 6
2
votes
1 answer
Fillna by relating multiple columns using a function
I have 3 columns in the dataframe. object, id and price. I want fill
the blanks by reading the id column and discover which price should I
use. For exemple: If the id ends in (A,B or C) the price should be 30
but if it's end (7A,7B or 7C) the…
Harry Jones
- 147
- 6
2
votes
1 answer
Fill na for a filter of dataframe
I have a dataframe similar to below:
col1 col2 col3 col4
0 101 1000 NaN NaN
1 102 2000 51 1500
2 103 2500 52 2800
3 104 3600 53 NaN
4 105 2400 NaN NaN
5 106 …
Meet
- 461
- 4
- 19
2
votes
2 answers
How to fill data gaps only when extremities have the same value, and limited to a maximum of occurrences?
I searched a lot here for an answer that could solve this but couldn't find. The desired result is to fill only gaps when the extremities are equal values, limited to lengths of 4 values:
My dataset:
0 NaN
1 NaN
2 NaN
3 5.0
4 …
User365Go
- 91
- 9
2
votes
1 answer
df.fillna() does not replace all NaN values
I'm trying to replace NaN values in my dataframe using:
values = {'pitch_type': 'UN', 'px': -1, 'pz': -1, 'pitch_type_prev': -1,'px_prev': -1,
'pz_prev': -1}
df_sample.replace(np.nan, values)
It fills all but two values. The returned…
Iris
- 21
- 2
2
votes
2 answers
Python: how to fill up the mean value referencing from another dataframe column
I have a housing dataframe:
where there are missing values in the Price column. I wish to fill the missing values by the mean price in the respective suburb.
This is my code for filling up the mean price by the same…
NatK
- 21
- 2
2
votes
2 answers
How to fillna limited by date in a groupby
I am working with the following Dataframe that has some NaN values inside.
df =…
2
votes
2 answers
python pandas stop fillna at last non NaN value
I have a dataframe where the index is date increasing and the columns are observations of variables. The array is sparse.
My goal is to propogate forward in time a known value to fill NaN but I want to stop at the last non-NaN value as that last…
JohnnieL
- 1,192
- 1
- 9
- 15
2
votes
0 answers
Incomplete filling when upsampling with `agg` for multiple columns (pandas resample)
I found this behavior of resample to be confusing after working on a related question. Here are some time series data at 5 minute intervals but with missing rows (code to construct at end):
user value total
2020-01-01 09:00:00…
Tom
- 8,310
- 2
- 16
- 36