Relating to up- or down-sampling time series data in the Python package pandas, specifically the methods pandas.DataFrame.resample() and pandas.Series.resample().
Questions tagged [pandas-resample]
314 questions
0
votes
1 answer
convert pandas list to 1 min ohlc data
I am trying to convert a streamed list of values that are appended to a pandas dataframe and convert them to a 1 min ohlc data.
I am having issues with trying to get the resample done and have hit a wall.
Here is my data.
Timestamp Price …
user1601716
- 1,893
- 4
- 24
- 53
0
votes
1 answer
Rolling up daily data to weekly with resample and offset in pandas
I have daily covid data that I would like to roll up to weekly. The problem is that I want my weeks to go from Sunday to Saturday, but the default is Mon to Sun. I tried to use loffset, but it's only changing the dates not my data, plus it's adding…
Jonathan Hay
- 195
- 11
0
votes
0 answers
ValueError: Inferred frequency None from passed values does not conform to passed frequency H
I have a pandas dataframe that I have reindexed to use hourly values. It is a list of weather observations that was a mix of 3-hourly, hourly, and more-than-once-per-hour data. Now I am trying to add some columns to calculate and plot the values I…
JudyD
- 1
- 1
0
votes
1 answer
Time Series Resampling with wrong out and without Frequency
At the moment I am working on a time series project.
I have Daily Data points over a 5 year timespan. In between there a some days with 0 values and some days are missing.
For example:
2015-01-10 343
2015-03-10 128
Day 2 of october is missing.
In…
Mannheimer_Coder
- 3
- 3
0
votes
1 answer
Pandas: Resample on duplicate timestamps
I am looking for a solution to resample gnss based data. There are two files in csv format. The data can be unambiguously identified via the time stamps and receivers. To enable more effective use in the database, the data must be reduced.
The data…
cminton
- 11
- 2
0
votes
0 answers
Using Python (Pandas) groupby and datetime to sum values in the morning and afternoon for different markets
I have a datetime column that looks like the following:
And a "market" column that has sting values such as ['NYC', LA', 'HOU', etc.] I also have a column "ID" that has random Order IDs. I am trying to look at each market by day (data goes back 5…
Kyle Reagan
- 19
- 4
0
votes
2 answers
Get rid of data if there are less than 3 data points in the hour
I am fairly new to this so please bear with me. I have a df where the index is in datetime format. My other columns are concentration and a count column that just consists of 1s.
Timestamp
Concentration
Count
2018-01-01…
sid
- 1
- 1
0
votes
0 answers
DataError: No numeric types to aggregate when using resampling in pandas dataframe
I have a list of dictionaries with values that I convert to a dataframe in pandas. I also convert the date-information (here measured in ms) to datetime and set a new index. The values are recorded every 5 seconds. When I want to resample them to 1…
PeterBe
- 700
- 1
- 17
- 37
0
votes
1 answer
Diurnal cycle of data
I have data recorded automatically every minute.
The data looks like this
The plot that I am trying to have should be like this.
I am trying to average each variable such that the final will have 1440 (60X24) time steps. In this way, I will able…
0
votes
1 answer
Pandas resample monthly data into custom frequency (seasonal) data
Background
I have a monthly dataset and want to resample it to seasonal by adding monthly data.
Seasonal refers to:
(Dec,Jan,Feb), (Mar,Apr,May),(June,July,Aug,Sep),(Oct,Nov)
The Data
dti = pd.date_range("2015-12-31", periods=11, freq="M")
df =…
Moon Child
- 102
- 5
0
votes
1 answer
How to resample an hourly time series to start at a specific hour and end exactly 24 hours later
I want to resample my dataframe including hourly precipitation values to daily (frequency of 24 hours) starting at a specific hour in the day (in my case it would start from 2020-02-01 06 UTC).
hourlydataframeimagefor2020-02-01:
I tried:
df =…
user16031555
0
votes
1 answer
How do I include other columns in dataframe half hourly to daily resample using Pandas?
I have a weather dataframe which is formatted in half hour intervals (simplified below):
df = pd.DataFrame({ 'date': ['2019-01-01 09:30:00', '2019-01-01 10:00', '2019-01-02 04:30:00','2019-01-02 05:00:00','2019-01-04 02:00:00'],
…
Jordan Ford
- 81
- 6
0
votes
1 answer
getting 'SeriesGroupBy' object has no attribute 'high' in pandas resample of crypto data
So when loading kraken.com historical datafeed into pandas to convert from tick data (time and sales) to OHLC (Open,High,Low,Close) data I run into an issue where I am not able to resample for 'high' & 'low' as they throw errors, but 'first',…
jamason1983
- 419
- 1
- 3
- 11
0
votes
1 answer
how to implement Resampling mean of dataframe object such that zero values are excluded in calculation of mean
I have a dataframe with timestamp values. I have been able to figure out how to use resample method of dataframe and applying function last() or mean() to results. I am doing it as follows…
lonstud
- 525
- 2
- 19
0
votes
0 answers
How to re-arrange a groupby object to a fixed lengh of 2D array?
This may be a very special conversion, but assuming I have a panadas DataFrame df:
import pandas as pd
df=pd.DataFrame({'a':[1,2,3,4,5,6,7,8,9],'b':[2,2,2,2,3,3,3,3,3]}
a b
0 1 2
1 2 2
2 3 2
3 4 2
4 5 3
5 6 3
6 7 3
7 8 3
8 9 …
John
- 41
- 1
- 6