Relating to up- or down-sampling time series data in the Python package pandas, specifically the methods pandas.DataFrame.resample() and pandas.Series.resample().
Questions tagged [pandas-resample]
314 questions
3
votes
1 answer
How do I split a dataframe based on datetimes differences?
Say I have this dataframe with datetimes separated by an unknown time interval:
data[0]:
mintime check
1375 2020-02-18 12:17:51.275000064+00:00 GO1
1376 2020-02-18 12:17:56.484999936+00:00 GO1
1377 …
Peter La Anguila
- 169
- 9
3
votes
1 answer
Pandas detect overday date
I have a data which looks like this:
Time
Data
13:45:00
Data 1
13:45:03
Data 2
13:45:14
Data 3
13:45:22
Data 4
13:45:24
Data 5
00:00:03
Data 6
00:00:26
Data 7
The data have over day data but it only contains time but no date,…
Hanyi Koh
- 327
- 1
- 4
- 15
3
votes
3 answers
Group if difference of datetime index is less than 5 minutes of a pandas series
I want to perform a groupby.first() of a pandas timeseries where the datetime index is almost consecutive, where almost is less than 5 minutes of difference.
I have seen a lot of material but never if the datetime is not consecutive like in my…
mat
- 181
- 14
3
votes
1 answer
How to do time series backward resampling e.g. 5 business days starting on the last data date?
I would like to compute weekly returns but starting from the end date backwards. This is my initial attempt to implement it using pandas:
import pandas as pd
import numpy as np
from pandas.tseries.offsets import BDay
index =…
SkyWalker
- 13,729
- 18
- 91
- 187
3
votes
1 answer
Python Pandas: How to use "resample" together with "idxmin"?
I have a dataframe with a pandas datetime index.
TIMESTAMP water
2020-06-24 13:50:00 -0.5
2020-06-24 14:00:00 -0.6
2020-06-24 14:10:00 -0.7
2020-06-24 14:30:00 -0.5
2020-06-24 14:40:00 -0.8
...
I want to get the index of the…
eetiaro
- 342
- 1
- 3
- 14
3
votes
0 answers
How to groupby or resample by a specific number of rows -- using Dask (Python)
I'm trying to downsample Dask dataframes by any x number of rows.
For instance, if I was using datetimes as an index, I could just use:
df = df.resample('1h').ohlc()
But I don't want to resample by datetimes, I want to resample by a fixed number of…
zippycorners
- 31
- 2
3
votes
1 answer
Plot a .resample(D).size() from 2 different years into one chart?
I have some data from 2019 and 2020 starting in March until the end of May for each year.
I've done this to the datetime
####Working with Date
df['Date']= pd.to_datetime(df['Date'])
df['Time_Hour'] = df['Date'].apply(lambda x:…
blankslatecoder
- 123
- 7
3
votes
2 answers
Resampling boolean values in pandas
I have run into a property which I find peculiar about resampling Booleans in pandas. Here is some time series data:
import pandas as pd
import numpy as np
dr = pd.date_range('01-01-2020 5:00', periods=10, freq='H')
df =…
Tom
- 8,310
- 2
- 16
- 36
2
votes
1 answer
Using Pandas to resample and combine data series with different timestamp indices
I'm attempting to use pandas to manipulate some data and not seeming to find a built-in way to do resample my data to merge datasets with differing time indices.
It's not hard to do what I want using loops and such, but I'm trying to see if there's…
Dakiltedyaksman
- 21
- 1
2
votes
2 answers
Resampling data from 6 min to 5min with nan
I have a linear interpolation problem with nans in my data. I have instantaneous measurements that I want to resample from 6 min intervals to 5 min intervals.
df = pd.DataFrame(zip(['10:00','10:06','10:12','10:18','10:24'],
[1,…
pyaj
- 545
- 5
- 15
2
votes
1 answer
Pandas resample method does not work properly
I have been analyzing the Seoul Bike Sharing Demand dataset, which is available at Seoul Bike Sharing Demand
. During my analysis, I found the need to use a resampling method. To accomplish this, I loaded the dataset into a Pandas DataFrame, which I…
M.Arya
- 145
- 1
- 4
2
votes
1 answer
Pandas.resample with min and max of each time period
I have the following dataframe (top 30 lines shown, its weather station data at 5 minute intervals, 2 years of data) and need to resample it into days with mean values for each column, I also need to get the min and max of 'hum' and 'temp' columns…
matt cooper
- 101
- 1
- 8
2
votes
1 answer
How can I make NaN values sum to NaN rather than 0 when using df.resample?
I have the following example dataframe:
>>> import pandas as pd
>>> import numpy as np
>>> d = {'date': pd.date_range(start='2022-12-09 00:00:00',
end='2022-12-09 02:50:00',
…
CDJB
- 14,043
- 5
- 29
- 55
2
votes
1 answer
Python Resample: How do I keep NaN as NaN?
When resample from monthly data to quarterly, I want my last value NaN to remain as NaN. How should I tweak my code?
Thank you
HS6P1
Jan1989 69.9
Feb1989 59.3
Mar1989 83.5
Apr1989 100.4
May1989 101.4
Jun1989 100.3
Jul1989 98
Aug1989 …
123456
- 393
- 3
- 12
2
votes
1 answer
Upsampling and dividing data in pandas
I am trying to upsample a pandas datetime-indexed dataframe, so that resulting data is equally divided over the new entries.
For instance, let's say I have a dataframe which stores a cost each month, and I want to get a dataframe which summarizes…
Clej
- 416
- 3
- 13