4

I'm trying to use pandas to resample 15 minute periods into 1 hour periods but by applying a custom function. My DataFrame is in this format;

Date                      val1       val2                  
2016-01-30 07:00:00       49.0       45.0
2016-01-30 07:15:00       49.0       44.0
2016-01-30 07:30:00       52.0       47.0
2016-01-30 07:45:00       60.0       46.0
2016-01-30 08:00:00       63.0       61.0
2016-01-30 08:15:00       61.0       60.0
2016-01-30 08:30:00       62.0       61.0
2016-01-30 08:45:00       63.0       61.0
2016-01-30 09:00:00       68.0       60.0
2016-01-30 09:15:00       71.0       70.0
2016-01-30 09:30:00       71.0       70.0

..and i want to resample with this function;

def log_add(array_like):
    return (10*math.log10((sum([10**(i/10) for i in array_like])))))

I do;

df.resample('1H').apply(log_add)

but this returns an empty df, doing this;

df.resample('1H').apply(lambda x: log_add(x))

does the same too. Anyone any ideas why its not applying the function properly?

Any help would be appreciated, thanks.

JayBe
  • 75
  • 1
  • 1
  • 5
  • I think you want to use `df.resample('1H').agg (...)` (don't quote me on it). Resample object is pretty much a groupby now. – pbreach Dec 23 '16 at 11:35

1 Answers1

4

You can add parameter on what is implemented in 0.19.0 pandas:

print (df.resample('1H', on='Date').apply(log_add))

Or set Date to index by set_index:

df.set_index('Date', inplace=True)
print (df.resample('1H').apply(log_add))

Also first check if dtype of column Date is datetime, if not use to_datetime:

print (df.dtypes)
Date     object
val1    float64
val2    float64
dtype: object

df.Date = pd.to_datetime(df.Date)

print (df.dtypes)
Date    datetime64[ns]
val1           float64
val2           float64
dtype: object
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
  • 1
    Thanks for the reply but this didn't work unfortunately; Date is already the index and the resample works for normal functions like 'sum' for example so i'm thinking my custom func is at fault but not sure why? – JayBe Dec 23 '16 at 12:12
  • @JasonB A couple points. Use `numpy` instead of `math` and if you are using python 2, your function won't work properly because of the integer division. rerun your code with `from __future__ import division` – Ted Petrou Dec 23 '16 at 12:44
  • Thanks for the points- duly noted. I'm currently using Python 3 too. – JayBe Dec 23 '16 at 13:34
  • I test it with pandas 0.19.1 and python 3 and for me it works perfectly. Problem is with samle also? – jezrael Dec 23 '16 at 13:37
  • @Ted Apologies, I've rerun this with np.log10 and it works perfectly! Thanks very much, that's great- i'd have been on this for ages otherwise. – JayBe Dec 23 '16 at 14:17
  • Doesn't seem to work with `.apply(func, args=(arg1,))` – Little Bobby Tables Jun 01 '17 at 14:32
  • What about `.apply(lambda x: func(arg1,))` ? – jezrael Jun 01 '17 at 14:35