Relating to up- or down-sampling time series data in the Python package pandas, specifically the methods pandas.DataFrame.resample() and pandas.Series.resample().
Questions tagged [pandas-resample]
314 questions
1
vote
1 answer
Difference between pandas aggregators .first() and .last()
I'm curious as to what last() and first() does in this specific instance (when chained to a resampling). Correct me if I'm wrong, but I understand if you pass arguments into first and last, e.g. 3; it returns the first 3 months or first 3 years.
In…
Dumb chimp
- 444
- 1
- 4
- 13
1
vote
2 answers
Resampling data by week
I have a time series dataset of 100 users looking at 365 days of transactions. The dataframe is currently set up like the following:
date 2018-04-01 2018-04-02 2018-04-03 2018-04-04 2018-04-05 ...
userid
1 0 0 …
rafvasq
- 1,512
- 3
- 18
- 48
1
vote
1 answer
Python Resample - Pad does not fill NAN
I am trying to fill in the NaN's after I upsample my timeseries with resample's pad() function.
I used the resample('1min').asfreq to upsample from hourly data to minute-interval data, then used resample.('1min').pad() it does not fill in the NaN…
Starbucks
- 1,448
- 3
- 21
- 49
0
votes
0 answers
Resample hourly timeseries to MS granularity with offset
This question can be seen as an extension of the following question:
Resample hourly TimeSeries with certain starting hour
I have the following dataframe:
import pandas as pd
index = pd.date_range(start='2023-01-01 17:00', periods=24*59, freq='H')…
Silverwilly
- 31
- 4
0
votes
1 answer
Efficient Resampling time series
I have been working on time-series data resampling using Pandas. It works well and give required results. However, the performance is little slow as per my current requirement.
Problem: I have minute data that I need to resample to many frequencies…
ap14
- 4,393
- 1
- 15
- 30
0
votes
0 answers
why pandas infer freq returns Daily when it is Hourly data
I have hourly data, when I infer it, it returns Daily data. I am surprized. My actual goal is converting this hourly data to daily mean.
df =…
Mainland
- 4,110
- 3
- 25
- 56
0
votes
1 answer
Dataframe how to groupby by a certain time duration
I have a pandas dataframe "df" with 2 columns named as
["patientVisit_id", "demandTime",],
where patientVisit_id is an int64 column, and demandTime is datetime.
I want to write a python function to group by the number of patientVisit_id each day…
MIMIGA
- 293
- 1
- 9
0
votes
1 answer
Check Dataframe Exactly at 10 min interval starting from 9.15
I am stuck on code , where I want to check dataframe every 10 min starting from 09:15.
If I use minute%10=0 then It checks at 9:20 and 9.30 which is not correct.
It should always check at 9.25 and 9.35 till 15.25.
Thanks
Checking at specific…
Mintoo
- 5
- 1
- 1
0
votes
1 answer
Resampling of a DataFrame by 1 Hour in pandas gives unexpexted NaN values
Resampling of a DataFrame by 1 Hour in pandas gives unexpected NaN values
I have a dataframe having 3 columns. 1st Column contains date ( like 2020-07-01,2020-07-01...); 2nd column contains time ( like 00:00:00, 01:00:00...) for one month on hourly…
ARINDAM DAS
- 3
- 3
0
votes
0 answers
How to resample data to monthly on 1. not on last day of month?
I have daily price data on Bitcoin and the USD/EUR. I resampled them to monthly data by
usd_df_m = usd_df.resample("M", on="Date").mean()
df_months = df.resample("M", on="Date").mean()
I also got data on the monthly federal funds rate. I tried to…
BlankerHans
- 49
- 8
0
votes
0 answers
Resampling multiindex dataframe with one of the index containing string values
I have a dataframe as follows
df:
ticker A B C D
Date symbol
2022-01-01 XYZ 0 1 0 0
JFK 0.5 0 0.5 0
2022-01-02 EFG…
MathMan 99
- 665
- 1
- 7
- 19
0
votes
0 answers
how to split data hourly in Pandas
i have a data resembling this
EndTime Duration PartsProduced
StartTime
2020-09-03 00:14:51 2020-09-03 00:46:56 1925 9100.0
2020-09-03 00:53:09…
PKD
- 1
- 1
0
votes
1 answer
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index', but my index is a DateTimeIndex
I have some time series data that is measured hourly, I want to average the values over different resolutions by resampling. I keep getting the following error:
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an…
Tom
- 109
- 9
0
votes
1 answer
How to speed up resample idxmax and idxmin column calculations in pandas?
I have 1 minute ohlcv data in a pandas dataframe.
I want to resample it with 5 minute intervals and see if the high of the 5 minutes was hit first, or the low, all the while keeping the ohlcv values.
Input is…
nurettin
- 11,090
- 5
- 65
- 85
0
votes
2 answers
Python resample to only keep every 5th day by group
I have a dataframe, consisting of daily stock observations, date and PERMNO (Identifier). I want to resample the dataframe to only consist of observations for every 5th trading day for every stock. The dataframe looks something like the…
Sebastian
- 27
- 5