Relating to up- or down-sampling time series data in the Python package pandas, specifically the methods pandas.DataFrame.resample() and pandas.Series.resample().
Questions tagged [pandas-resample]
314 questions
2
votes
1 answer
How to resample value in pandas column?
I know about the resample function on time-series data. I want something similar on a normal column with 3000 examples. I want to keep the length. I want every row to have the value of the last occurrence in a n- long window.
I know about group by…
Borut Flis
- 15,715
- 30
- 92
- 119
2
votes
1 answer
Pandas linear interpolation for geometrical X-Y data seems to ignore points
I am trying to upsample my dataframe in pandas (from 50 Hz to 2500 Hz). I have to upsample to match a sensor that was sampled at this higher frequency. I have points in x, y, z coming from a milling machine.
When I am plotting the original data the…
Federica Tomola
- 57
- 4
2
votes
1 answer
Python pandas resample to period that doesn't subdivide into a 24 hr period
I have a pandas dataframe with high frequency (second or minutely) stock data.
e.g.
Date Time Open ... Low Close Volume
DT ... …
AaronE
- 53
- 5
2
votes
1 answer
Pandas interpolation giving odd results
I am using Pandas to interpolate datapoints in time, however when resampling and interpolating, I get different results for the same interpolated time when using different resampling rates.
Here is a test example:
import pandas as pd
import…
mystery_lektro
- 75
- 5
2
votes
1 answer
Group by and resample in specified time interval in Python
How can I resample for df below at 1min frequency using forward fill ffill and backward fill bfill for each id(using groupby('id')) for the time interval 2017-01-01 00:00:00 and 2017-01-05 00:00:00, ie., the first timestamp is 2017-01-01 00:00:00…
nilsinelabore
- 4,143
- 17
- 65
- 122
2
votes
1 answer
Pandas: how to get index value of non-unique index
I have a data frame with a date time index where index values are non unique (see last two index values).
I would like to get the next valid index value given a time delta of +5 seconds from the first index value. In the case below, the first index…
finstats
- 1,349
- 4
- 19
- 31
2
votes
2 answers
Mean Aggregations using pandas GroupBy and Time Series resampling
I'm having trouble with Pandas groupby functionality and Time Series. I've read the documentation, but I can't see to figure out how to apply aggregate functions to multiple columns and calculate the mean of the volume (average) of the „aggregate “…
udi76623
- 23
- 5
2
votes
0 answers
Incomplete filling when upsampling with `agg` for multiple columns (pandas resample)
I found this behavior of resample to be confusing after working on a related question. Here are some time series data at 5 minute intervals but with missing rows (code to construct at end):
user value total
2020-01-01 09:00:00…
Tom
- 8,310
- 2
- 16
- 36
2
votes
1 answer
pandas .resample() does not work with some examples. Why?
Question
I want the result of Pandas .resample() method to have 2 levels of index. As you can check it out on the example below, the resampling of good_example_df produces the 2 level of index.
When I changed the 2nd element of index_list from…
Eiffelbear
- 399
- 1
- 4
- 23
2
votes
1 answer
How to do a "weighted upsampling" in `pandas` `Series`?
I have a function in python that I use to distribute values when upsampling. For example, to upsample distances my car's driven from per month to per day:
def distribute(df, freq:str):
# if there's an easier way please do comment
df_new =…
ElRudi
- 2,122
- 2
- 18
- 33
1
vote
1 answer
Define a 'minimum' for pandas.DataFrame.resample() that is lower than current dataframe's minimum TimedeltaIndex
I have a dataframe for a protocol that tracks the value of 2 settings every minute of a test. For example:
In [1]: df = pd.DataFrame(
{
"time": [
pd.Timedelta(1, unit="min"),
…
Braden
- 680
- 1
- 11
- 26
1
vote
1 answer
Python StatsModels: ValueError: Expected frequency D. Got M
I am using statsmodels.graphics to draw a month_plot from timeseries data in a kaggle dataset. I have converted the data to daily frequency mean data as required for the plot. However, I am getting an error that says the expected data frequency is…
Mainland
- 4,110
- 3
- 25
- 56
1
vote
1 answer
Pandas - How to merge rows based on a criteria
I have a Pandas dataframe with thousands of rows that I want to combine to smaller number of rows.
In the new dataframe, I want:
Certain columns be summation of grouped rows
Some columns be average of grouped rows, and
Some other columns be…
MSN
- 13
- 2
1
vote
1 answer
Downsampling with pandas produces NaN
I am downsampling data from 15 minutes scale to hourly scale with pandas resample.mean(). The result has NaN values.
Data
C_hourly
Data = Data.set_index('readable_time')
C_hourly = Data.groupby(['sensor_no'],as_index=False).resample('H').mean()
I…
Scholar7
- 11
- 1
1
vote
1 answer
resample('D').interpolate() fills value, but resample('Y').interpolate() produces nans?
Let's start with two dates, two days apart, resample daily, and interpolate:
In [1]: ts = pd.Series([1, 2], index=pd.DatetimeIndex(['1950-01-01', '1950-01-03']))
In [2]: ts.resample('D').interpolate()
Out[2]:
1950-01-01 1.0
1950-01-02 …
ignoring_gravity
- 6,677
- 4
- 32
- 65