Relating to up- or down-sampling time series data in the Python package pandas, specifically the methods pandas.DataFrame.resample() and pandas.Series.resample().
Questions tagged [pandas-resample]
314 questions
0
votes
1 answer
ValueError: Length of new names must be 1, got 2
I am trying to resample a time series, with reference to the start and end data in another table as below. Data is as below:
df1:
Index Timestamp Data ID
2 1 2 2014-03-04 13:16:44.310 30.0 2
6 8 …
nilsinelabore
- 4,143
- 17
- 65
- 122
0
votes
1 answer
What is the fastest way to repeatedly resample timeseries data of the same shape from hourly to yearly in python
What is the fastest way to repeatedly resample timeseries data of the same shape?
Problem: I have 30 years of an hourly timeseries which I want to resample to yearly and by calendar year (resample rule 'AS'). I need to both find the mean in each…
Mike Twomey
- 13
- 3
0
votes
1 answer
Resample time series based on different dates
I have a table df1 which consists of multiple time series represented by different ID. I want to resample the time series for each ID based on the start and end dates in another table df2. df1 and df2 are as below:
df1:
Index …
nilsinelabore
- 4,143
- 17
- 65
- 122
0
votes
1 answer
Get Last Value of Semi Annual and Financial Year End
Let's generate a df with random numbers using code below.
test = pd.DataFrame(index=pd.date_range('2019-03-28', '2020-07-31'))
np.random.seed(101)
test['Value'] = np.random.rand(test.shape[0])*1000
If I resample on quarterly basis and get the last…
Lopez
- 461
- 5
- 19
0
votes
1 answer
Python Pandas Upsampling on average values between data points (15min to 1min)
i have some issues with my dataresampling in pandas. I´m trying to upsample 15 min values to 1min values. The resampled dataframe values shoud contain the sum spliited equaly between the two values of the original dataframe. This codes generates an…
Lukas
- 3
- 2
0
votes
1 answer
Pandas rearrange and interpolate time-series based with datetime index
I have a recurrent problem that I solve inelegantly every-time, and I am unable to find a good way to do it.
Let's say I have a dataframe with datetime in index, spanning every 3 hours (df1). I have another dataframe spanning every day (df2).
I want…
Nihilum
- 549
- 3
- 11
0
votes
0 answers
How to convert the recorded data on (10Hz) high frequency to (1Hz) low frequency using python?
I want to convert the data which is recorded in 10Hz into 1Hz. I saw some solutions and i tried it, DO i need to use resample function?, do i need to convert timestamp into sec or minute something?
My question is I have data in millions of rows when…
SSS
- 73
- 11
0
votes
0 answers
Differences in calculated mean between groupby and groupby + resample
I'm trying to do weekly averages from meteorological time series data with several values per day. I found two ways of doing it, one with groupby and resample and the other one with groupby alone (based on this) but they give slightly different…
irmavar
- 1
- 2
0
votes
1 answer
group by with resample has column is being duplicated
Background: I have covid data that I aggregated to the state level. Then I aggregated my days to the week level (this works). However, when I run the day to week logic with almost exactly the same data just at the county level I get an error. More…
Jonathan Hay
- 195
- 11
0
votes
1 answer
why does xarray resample mean calculation create an extra time?
I'm loading an annual dataset with xarray. I'd like to create 35-year means of the data (not a rolling average, just chunked into 35 year means).
Right now I do the following:
filename =…
user14771692
- 43
- 6
0
votes
1 answer
Daily to Weekly Pandas conversion
I am trying to convert my 15ys worth of daily data into weekly by taking the mean, diff and count of certain features. I tried using .resample but I was not sure if that is the most efficient way.
My sample data:
Date,Product,New…
Ksh
- 85
- 4
0
votes
1 answer
Python pandas resample 15 minutes ohlc to 75 minutes ohlc
Below is the dataframe for one of the stock in indian market.
open high low close volume
date
2021-06-17 09:15:00+05:30 815.55 819.15 807.05 818.45 54372
2021-06-17…
Vilas
- 1,695
- 1
- 13
- 13
0
votes
1 answer
the ValueError: cannot reindex a non-unique index with a method or limit appears for just cryptocurrency
Would you please tell me what is wrong with the following as I get the error:
ValueError: cannot reindex a non-unique index with a method or limit
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import pandas_datareader as…
john
- 27
- 6
0
votes
0 answers
Unable to resample the pandas with date column. TypeError Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index'
The dataframe is
Date Amount
2019-07-01T00:00:00+05:30 2055359.98
2019-07-01T00:00:00+05:30 2055359.98
2019-07-01T00:00:00+05:30 145198200.0
2019-07-01T00:00:00+05:30 145198200.0
2019-07-02T00:00:00+05:30 …
Nithin Reddy
- 580
- 2
- 8
- 18
0
votes
1 answer
Pandas Dataframe time series resample, how to modify bins to fit underlying dataset start and end time
I excercice with some stock market data and have a dataframe starts at 09:30 and ends at 16:00. I want to resample to an 4Hour Interval using
agg_dict = {'open': 'first','high': 'max','low': 'min','cls': 'last','vol': 'sum'}
data_4hour =…
Hank Gordon
- 127
- 1
- 9