pandas dataframe NaN when reindexing with period_range

Question

I want the code to add new rows not present in the dataframe. I'm getting NaN values when reindex-ing with period_range. I get the correct period_range but NaNs instead of preserving available values for column 'A'. Below is shown the code example:

I guess the problem comes out because using PeriodIndex and DatetimeIndex objects.

                      A
2018-10-31 14:08:26 NaN
2018-10-31 14:08:27 NaN
2018-10-31 14:08:28 NaN
2018-10-31 14:08:29 NaN
2018-10-31 14:08:30 NaN


import pandas as pd

data=[['2018-10-31 14:08:26', 1],
      ['2018-10-31 14:08:28', 2],
      ['2018-10-31 14:08:30', 3]]

df = pd.DataFrame(data=data, columns=['time','A'])
df.time = pd.to_datetime(df.time)
ts = df.time
idx = pd.period_range(min(ts), max(ts),freq='s')
df = df.set_index('time',drop=True)
df = df.reindex( idx )

score 2 · Accepted Answer · answered Mar 05 '19 at 17:05

data = [['2018-10-31 14:08:26', 1],
        ['2018-10-31 14:08:28', 2],
        ['2018-10-31 14:08:30', 3]]

df = pd.DataFrame(data=data, columns=['time','A'])
df['time'] = pd.to_datetime(df['time'])
df.set_index('time').resample('S').asfreq()

Output

>>> df
                        A
time    
2018-10-31 14:08:26     1.0
2018-10-31 14:08:27     NaN
2018-10-31 14:08:28     2.0
2018-10-31 14:08:29     NaN
2018-10-31 14:08:30     3.0

score 0 · Answer 2 · answered Mar 05 '19 at 16:46

It's required to change DatetimeIndex to PeriodIndex:

df = df.set_index('time',drop=True)
df.index=df.index.to_period('S')
df = df.reindex( idx )

                       A
2018-10-31 14:08:26  1.0
2018-10-31 14:08:27  NaN
2018-10-31 14:08:28  2.0
2018-10-31 14:08:29  NaN
2018-10-31 14:08:30  3.0

pandas dataframe NaN when reindexing with period_range

2 Answers2