Relating to up- or down-sampling time series data in the Python package pandas, specifically the methods pandas.DataFrame.resample() and pandas.Series.resample().
Questions tagged [pandas-resample]
314 questions
0
votes
1 answer
Hourly mean after resampling and subsetting the dataset
I have multiple .nc files that I combined using xr.open_mfdataset, floowed by subsetting it for study region and applying the mask using regionmask and resampled it to take hourly mean, but it is giving wrong answer.
ds…
Yash_U
- 1
- 2
0
votes
0 answers
Finding hourly mean over multiple dataset
I want to take the hourly mean of t2m values after loading multiple .nc files, loaded using xr.open_mfdataset, but after resampling and taking the mean gives incorrect time dimensions.
I have loaded multiple .nc files using
ds…
Yash_U
- 1
- 2
0
votes
2 answers
Resampled series with timedeltas does not start at zero
I would like to resample a pandas series with timedelta starting with the zero interval (timedelta(seconds=0)) instead of the first occurrence.
For example, for
from datetime import timedelta
from random import sample
s = pd.Series(1,…
Michael Dorner
- 17,587
- 13
- 87
- 117
0
votes
1 answer
monthly aggregation with custom day
I have a dataframe like this,
tidx = pd.date_range('2022-10-01', periods=10, freq='10D')
data_frame = pd.DataFrame(1, columns=['inventory'], index=tidx)
print(data_frame)
Output:
inventory
2022-10-01 1
2022-10-11 …
Phoenix
- 359
- 7
- 15
0
votes
2 answers
Pandas Resample creating new hourly data
I am using the resample funtion to go from minute data to hour data. The issue is my original DF has only from 10:30 to 15:59 data and the new resampled data is creating new hour data. How to I get rid of this data or have it resample only the time…
svlad
- 73
- 8
0
votes
2 answers
pandas resample with origin and closed status
I have a dataframe for inventory like the following,
tidx = pd.date_range('2022-10-01', periods=15, freq='D')
data_frame = pd.DataFrame(1, columns=['inventory'], index=tidx)
data_frame.iloc[-2:] = 0
print(data_frame)
inventory
2022-10-01 …
Phoenix
- 359
- 7
- 15
0
votes
0 answers
Value Error: index must be monotonic increasing or decreasing while using resample('M') function on datetime values
I am stuck at this point in my code. I am trying to divide the startdate and enddate into multiple rows based on months and for that I am trying to use the resample function to sample the dates on monthly basis. The sample code looks like this-
PS-…
0
votes
0 answers
Pandas dataframe.resample and mean gives higher values with increasing sample width
I have a time series of rain intensity (in µm/s), which I resample to 1 minute intervals. The data already has a 1 minute time step, but I may have data outage due to quality checks or basic equipment failure. The resample ensures that I have a…
karga
- 41
- 3
0
votes
1 answer
Values of Pandas DataFrame merged after resampling
I have tried to resample a dataframe from minutes to daily. For this I used the following lines:
df_1d = df.resample('D')['Value'].sum().asfreq(freq='D').reset_index()
df_1d.set_index(pd.DatetimeIndex(df_1d['DateTime']), inplace=True)
df_1d =…
Grinorye
- 23
- 4
0
votes
0 answers
Resampling time series data on a large number of columns
I have a data frame with over 180 columns that I need to sum over a time interval. The below resample code works fine when I enter the column names directly but not in a loop. Because of the large number of columns, I cannot write each of the column…
Abir
- 57
- 5
0
votes
1 answer
Resampling backwards in Python
I have been looking for a way to resample a dataframe backwards in time. So the last row is included in the result.
Imagine this is my dataframe
Close Volume
Datetime
2022-09-01…
Ivo
- 21
- 4
0
votes
0 answers
split / resample sales between dates to monthly
I need to split and resample sales by shops between dates to monthly (calendar month).
Offer
Start Date
End Date
Sales Shop 1
Sales Shop 2
Sales Shop 3
Total…
Pinaak
- 23
- 4
0
votes
0 answers
How to interpolate the values of a dask dataframe using upsampling method?
I have a very large dataframe (~1 million rows) made by dask.dataframe method in the following format (note that 'Timestamp' column is actually the index column of the dataframe' and that the seconds are truncated as a result of copy and paste from…
Roosha
- 65
- 1
- 5
0
votes
1 answer
Downsample dataframe correctly
I am having a input dataset, see an input sample bellow, and I want to downsample it. To do so I am using
resample_time=25
init_len = len(df.index)
df = df.set_index('time', drop=False).resample('{}S'.format(resample_time)).last().dropna()
df.index…
Darkmoor
- 862
- 11
- 29
0
votes
1 answer
Resample & calculate mean
Working with pandas in Python.
My data looks like:
2018-03-16 5.0
2018-03-17 5.0
2018-03-18 5.0
...
2018-03-31 5.0
After using
resample('MS').mean()
I get the following result:
2018-03-01 5.000000
The correct result should be…
n992
- 1
- 1