Pandas series datetimes to timedeltas (seconds)

Question

My Pandas dataframe has a sorted column of datetimes:

print(df.Time)

returns

0      2019-10-30 13:14:49
1      2019-10-30 13:15:25
2      2019-10-30 13:32:44
               ...        
997    2020-02-04 13:53:35
998    2020-02-04 14:22:46
999    2020-02-04 14:52:43
Name: Time, Length: 1000, dtype: datetime64[ns]

The very simple thing I'm attempting is to derive an array of timedeltas. I've tried:

df.Time[1:-1] - df.Time[0:-2]

which results in:

0         NaT
1      0 days
2      0 days
        ...  
996   0 days
997   0 days
998      NaT
Name: Time, Length: 999, dtype: timedelta64[ns]

The resulting length is correct, but I'm a little confused by the result.

It seems this is not the way to perform an operation on 2 subsets of a dataframe.

What is the correct approach, and is there a builtin method that produces timedeltas from a sorted column of datetimes?

Intended output looks something like:

0      35 seconds
1      1879 seconds
2      1720 seconds
        ...  
996    1805 seconds
997    1854 seconds
998    1791 seconds

you could try the shift function if you want to manipulate between rows. https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.shift.html — sammywemmy, Feb 05 '20 at 08:38

score 2 · Accepted Answer · answered Feb 05 '20 at 08:52

Sample DataFrame


0   2019-10-26 13:14:49
1   2019-10-30 13:16:49
2   2019-10-30 13:23:49
3   2019-10-30 13:32:49
4   2019-10-30 13:34:49
5   2019-10-30 13:45:49
6   2019-10-30 13:56:49
Name: Time, Length: 7, dtype: datetime64[ns]

You can simply use the pandas inbuilt diff function which calculates the difference of a DataFrame element in the same column of the previous row.

df.Time.diff()

The following command will result in:


0               NaT
1   4 days 00:02:00
2   0 days 00:07:00
3   0 days 00:09:00
4   0 days 00:02:00
5   0 days 00:11:00
6   0 days 00:11:00
Name: Time, dtype: timedelta64[ns]

score 0 · Answer 2 · answered Feb 05 '20 at 08:51

As sammywemmy said, you need:

df1['delta'] = df1.Time - df1.Time.shift()

On dummy dataframe:

df1.head(15)
Out[50]: 
                  Time      delta
0  2019-10-30 13:15:55      NaT
1  2019-10-30 13:16:11 00:00:16
2  2019-10-30 13:16:27 00:00:16
3  2019-10-30 13:16:54 00:00:27
4  2019-10-30 13:17:22 00:00:28
5  2019-10-30 13:17:23 00:00:01
6  2019-10-30 13:17:29 00:00:06
7  2019-10-30 13:17:44 00:00:15
8  2019-10-30 13:17:46 00:00:02
9  2019-10-30 13:17:48 00:00:02
10 2019-10-30 13:18:47 00:00:59
11 2019-10-30 13:18:52 00:00:05
12 2019-10-30 13:18:53 00:00:01
13 2019-10-30 13:18:59 00:00:06
14 2019-10-30 13:19:07 00:00:08

Pandas series datetimes to timedeltas (seconds)

2 Answers2