Relating to up- or down-sampling time series data in the Python package pandas, specifically the methods pandas.DataFrame.resample() and pandas.Series.resample().
Questions tagged [pandas-resample]
314 questions
2
votes
2 answers
Pandas rolling window statistics calculation with input data with uneven timestamps
quick background:
this can relate to pandas rolling, resample, asfreq, fillna
this is based on processing timeseries data so I want to use pandas offset (e.g., '1T', '5min', etc.) as an input to methods.
also, I'm applying forward looking window…
geekshock
- 31
- 3
2
votes
2 answers
Is there a way to vectorize adding missing months using resample?
I am trying to add missing months for each ID. Added months should have info on ID and year_month, and NaN for Product. My code achieves this using apply(), but is slow -- I am looking for a vectorized version, which can run significantly…
Dudelstein
- 383
- 3
- 16
2
votes
1 answer
Resampling datetime by date in pyspark
I am trying to use resample technique of pandas in pyspark but can't come to any conclusion.
+----------------------------+----+
| date |…
sargupta
- 953
- 13
- 25
2
votes
2 answers
How to resample ohlc data properly in pandas / custom fill method per column
I have got OHLC data with missing time frames. Suppose I have the following pandas dataframe denoted by the variable df:
Open High Low Close
2019-04-19 00:00:00 0.67068 0.67123 0.67064 0.67123
2019-04-19…
freak11
- 341
- 3
- 16
2
votes
0 answers
pandas resampling: aggregating monthly values with offset
I work with monthly climate data (e.g. monthly mean temperature or precipitation) where I am often interested in taking several-month means e.g. December-March or May-September. To do this, I'm attempting to aggregrate monthly time series data using…
2
votes
2 answers
How to resample frequency conditional on two columns in pandas?
I have the following dataframe:
import pandas as pd
data_as_dict = {'Date': {0: '2015-01-01 00:00:00', 1: '2015-01-01 00:00:02', 2: '2015-01-01 00:00:02', 3: '2015-01-01 00:00:02', 4: '2015-01-01 00:00:02',
5: '2015-01-01…
Rollo99
- 1,601
- 7
- 15
2
votes
1 answer
pandas resample - 5 minute blocks (not every 5th minute of the hour)
I have some data taken every minute and I want to resample it in 5 minute segments.
df.resample("5T").mean()
This has the effect of resampling to every fifth minute of the hour. i.e
12:00,12:05,12:10,12:15 etc.
What if my last data point was…
Lewis Morris
- 1,916
- 2
- 28
- 39
2
votes
2 answers
resampling raises ValueError: Values falls before first bin
I don't understand when and why this error is raised.
From my understanding, resample should create as many bins as needed in order to bin all the timestamps of the index. So the message "Values falls before first bin" does not make much sense to…
actual_panda
- 1,178
- 9
- 27
2
votes
1 answer
resampling with origin='end_day'
I don't understand what origin='end_day' does.
The docs give the following example:
>>> start, end = '2000-10-01 23:30:00', '2000-10-02 00:30:00'
>>> rng = pd.date_range(start, end, freq='7min')
>>> ts = pd.Series(np.arange(len(rng)) * 3,…
actual_panda
- 1,178
- 9
- 27
2
votes
1 answer
How to resample to a coarser resolution but to samples within the original index?
I have the following use case:
import pandas as pd
import numpy as np
# create dataframe
df = pd.DataFrame(data=np.random.rand(10, 3),
columns=['a', 'b'],
index=pd.date_range('2021-01-01', periods=10,…
SkyWalker
- 13,729
- 18
- 91
- 187
2
votes
1 answer
resampling a pandas dataframe and filling new rows with zero
I have a time series as a dataframe. The first column is the week number, the second are values for that week. The first week (22) and the last week (48), are the lower and upper bounds of the time series. Some weeks are missing, for example, there…
ojp
- 973
- 1
- 11
- 26
2
votes
0 answers
Using pd.resample with PeriodIndex
I have a time series with a resolution of 5 min. The entries represent an amount of energy used in the preceding 5 minutes. I. e. the entry for 00:25 gives the energy usage from 00:20:01-00:25:00.
import pandas as pd
idx =…
Durtal
- 1,063
- 3
- 11
2
votes
1 answer
How to down sample a dataframe in Python based on condition
I am new here so don't know how to use this site.
I have a timeseries data of 37404 ICU Patients. Each patient have multiple rows. I want to down sample my dataframe and select only 2932 patients (all rows of the respective patient ID). Can anyone…
user17416440
2
votes
2 answers
How to resample pandas to hydrologic year (Sep 1 - Aug 31)
I'd like to analyze some daily data by hydrologic year: From 1 September to 31 August. I've created a synthetic data set with:
import pandas as pd
t = pd.date_range(start='2015-01-01', freq='D', end='2021-09-03')
df = pd.DataFrame(index =…
mankoff
- 2,225
- 6
- 25
- 42
2
votes
0 answers
Does pandas resample changes sort order of the datetime index? How can I avoid this?
Hi have a dataframe with datetimeindex showing latest first and oldest last.
When using .resample.agg the order of the index turns around. I couldn't read anything in the docs. Why is that and how can I avoid/change it? Thank you.
Here I create an…
Hank Gordon
- 127
- 1
- 9