0

I have hourly data, when I infer it, it returns Daily data. I am surprized. My actual goal is converting this hourly data to daily mean.

df = pd.read_csv('/kaggle/input/hourly-energy-consumption/DOM_hourly.csv')
print(df.head())
print(df.shape)
df.set_index('Datetime', inplace=True, drop=True)
df.index = pd.to_datetime(df.index, format='%Y-%m-%d %H:%M:%S')
# drop duplicated index
df = df[~df.index.duplicated(keep='first')]
# Infer frequency of the data
# print("Data frequency is : %s" % pd.infer_freq(df.index))
df = df.asfreq(pd.infer_freq(df.index))
print(df.head(5))
print(df.shape)

Present output:

Datetime  DOM_MW
0  2005-12-31 01:00:00  9389.0
1  2005-12-31 02:00:00  9070.0
2  2005-12-31 03:00:00  9001.0
3  2005-12-31 04:00:00  9042.0
4  2005-12-31 05:00:00  9132.0
(116189, 2)


                     DOM_MW
Datetime                   
2005-05-01 01:00:00  7190.0
2005-05-02 01:00:00  6897.0
2005-05-03 01:00:00  7288.0
2005-05-04 01:00:00  7052.0
2005-05-05 01:00:00  7250.0
(4628, 1)
Mainland
  • 4,110
  • 3
  • 25
  • 56
  • cannot reproduce with given example data – FObersteiner Jun 15 '23 at 11:48
  • 1
    Your data come from https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption?select=DOM_hourly.csv? – Corralien Jun 15 '23 at 12:37
  • @Corralien correct – Mainland Jun 15 '23 at 21:07
  • For me the infer_freq is None because there are some missing dates. However, you don't need to set the frequency to resample your dataframe. – Corralien Jun 15 '23 at 21:09
  • @Corralien For me it looked bizarre when it automatically converter the data to monthly frequency before I even asked to do it. – Mainland Jun 15 '23 at 21:14
  • 1
    Did you try `df.resample('D').mean()` – Corralien Jun 15 '23 at 21:20
  • @Corralien You are correct. I did use that one `ddf = df.resample(rule='24H', kind='interval').mean().to_period('d')` This did the job perfectly. NOw I consider this question got answered. But I got one more issue. Here is a the new question: https://stackoverflow.com/questions/76485620/python-statsmodels-valueerror-expected-frequency-d-got-m – Mainland Jun 15 '23 at 21:28

0 Answers0