Use this tag for pandas.DataFrame.fillna or pandas.Series.fillna
Fill missing values using the specified method for a DataFrame or a Series with pandas (see also interpolate, ffill, bfill)
Docs:
Questions tagged [fillna]
528 questions
3
votes
1 answer
How do i df.fillna with category median values
I have a large dataset ~1mln rows, and about 5000 absent coordinates(i'd like to fill them with median value by category 'city'everything but fillna is working, how to make it happen?
city = ['London', 'Paris', 'Vienna', 'Milan','London', 'Paris',…
whocares
- 45
- 5
3
votes
2 answers
Python fillna with Series sharing same index
I have some null values on on column of my dataframe. I used a linear regression to predict the missing values but now I want to replace nan by the predicted value.
I would like to use the index as condition to fillna beacause I don't want all the…
Lilly_Co
- 169
- 12
3
votes
1 answer
pandas fillna sequentially step by step
I have dataframe like as below
Re_MC,Fi_MC,Fin_id,Res_id,
1,2,3,4
,7,6,11
11,,31,32
,,35,38
df1 = pd.read_clipboard(sep=',')
I would like to fillna based on two steps
a) First, compare only Re_MC and Fi_MC. If a value is missing in either of…
The Great
- 7,215
- 7
- 40
- 128
3
votes
2 answers
Fill missing values using a nested dictionary
Here is my sample dataframe:
df = pd.DataFrame(data=[[3, np.nan, np.nan],[5, np.nan, np.nan]], index=['country1', 'country2'], columns=[2021, 2022, 2023])
Here is my sample dictionary:
d = {'country1': {'key1': 'a', 'key2': 'assumed','key3': {2022:…
ric
- 153
- 8
3
votes
2 answers
pandas: fillna with data from another dataframe, based on the same ID and keeping all values
I want to fillna of df1, using df2, based on same colorID while keeping all rows and columns of df1.
df1=
colorID age flower
red1 12 sun
red2 na sun
green 23 hydro
red3 na hydro
yellow 3 sun
red4 na …
Kevin
- 31
- 3
3
votes
1 answer
fillna in Pandas - how to choose the best method automatically?
Suppose I have a dataframe that contains columns with lots and lots of nan values - in fact most values are none, except one (or a few that are identical), but are distributed along different lines. For example:
df = pd.DataFrame({'A':[np.nan, 2,…
durbachit
- 4,626
- 10
- 36
- 49
3
votes
1 answer
Unable to fill missing values with column value across all columns
I have a dataframe like as shown below
df = pd.DataFrame({'Credit_History':['Yes','ABC','DEF', 'JKL'],
'Loan_Status':['T1','T2',np.nan,np.nan],
'subject_status':['DUMMA','CHUMMA',np.nan,np.nan],
…
The Great
- 7,215
- 7
- 40
- 128
3
votes
1 answer
Partial fillna with condition python
I have the following data frame which I want to apply bfill as follows:
'amount'
'percentage'
Nan
1.0
20
2.0
10
Nan
Nan
Nan
Nan
3.0
50
4.0
10
Nan
5.0
10
I want to bfill Nan in the amount column as per percentage…
Mayur Zambare
- 31
- 3
3
votes
4 answers
Replace NaN with random number
I have a dataframe of numbers such as:
A B
2019-10-31 0.035333
2019-10-31 NaN
2019-11-30 -0.108532
2019-11-30 -0.030604
2019-11-30 NaN
I want to replace the NaN's in column B with a random gaussian number:
from random import…
t.pellegrom
- 313
- 3
- 10
3
votes
3 answers
Can I fill NaN-Values of one column with specific list elements of another column?
I have following dataframe (called items) for example:
| index | itemID | maintopic | subtopics |
|:----- |:------:|:---------:| ------------------:|
| 1 | 235 | FBR | [FZ, 1RH, FL] |
| 2 | 1787 | NaN | [1RH,…
leolumpy
- 63
- 6
3
votes
1 answer
Filling empty cells in python (import dataset from excel)
I have a dataset where I want to fill the columns and rows in python as shown below:
Dataset:
| P | Q |
|678|1420|
|678|---|
|609|---|
|583|1260|
|---|1260|
|---|1261|
|---|1262|
|584|1263|
|---|403|
|---|---|
Expected Result:
| P | Q…
Dhruv
- 43
- 4
3
votes
2 answers
Why pyspark fillna does not fill boolean values
I have a dataframe with a boolean column and I want to fill the missing values with False.
However, when I use fillna method, nothing happens:
df = spark.createDataFrame([(True,), (True,), (None,), (None,)], ['col'])
df.fillna(False).show()
The…
Grigory Sharkov
- 121
- 1
- 8
3
votes
1 answer
Stopping a forward fill based on a different string column changing in the DF
i'm new to Python so go easy!
I have a Dataframe like the following. I would like to forward fill the NaN's in the shares_owned column but stop when the string in df['ticker'] changes. And only start when another number appears in shares_owned…
Barnaby Cooper
- 43
- 5
3
votes
2 answers
Fill NaN values of DataFrame with random values from the column, depending on frequency
I am trying to fill a pandas dataframe NAN using random data of every column, and that random data appears in every column depeding on its frecuency. I have this:
def MissingRandom(dataframe):
import random
dataframe =…
Deco1998
- 101
- 6
3
votes
1 answer
How to do pd.fillna() with condition
Am trying to do a fillna with if condition
Fimport pandas as pd
df = pd.DataFrame(data={'a':[1,None,3,None],'b':[4,None,None,None]})
print df
df[b].fillna(value=0, inplace=True) only if df[a] is None
print df
a b
0 1 4
1 NaN NaN
2 3 …
ravijprs
- 39
- 1
- 4