1

I have a pandas dataframe with an unusual DatetimeIndex. The frame contains daily data (end of each day) from 1985 to 1990 but some "random" days are missing:

DatetimeIndex(['1985-01-02', '1985-01-03', '1985-01-04', '1985-01-07',
           '1985-01-08', '1985-01-09', '1985-01-10', '1985-01-11',
           '1985-01-14', '1985-01-15',
           ...
           '1990-12-17', '1990-12-18', '1990-12-19', '1990-12-20',
           '1990-12-21', '1990-12-24', '1990-12-26', '1990-12-27',
           '1990-12-28', '1990-12-31'],
          dtype='datetime64[ns]', name='date', length=1516, freq=None)

I often need operations like shifting an entire column such that a value that is at the last day of a month (which could e.g. in my DatetimeIndex be '1985-05-30') is shifted to the last day of the next (which could e.g. my DatetimeIndex be '1985-06-27').

While looking for a smart way to perform such shifts, I stumbled over Offset Aliases provided by pandas.tseries.offsets. It can be observed that there are the aliases custom business day frequency (C) and custom business month end frequency (CBM). When looking at an example, it seems like that this could provide exactly what I need:

mth_us = pd.offsets.CustomBusinessMonthEnd(calendar=USFederalHolidayCalendar())
day_us = pd.offsets.CustomBusinessDay(calendar=USFederalHolidayCalendar())
df['Col1_shifted'] = df['Col1'].shift(periods=1, freq = mth_us) # shifted by 1 month
df['Col2_shifted'] = df['Col2'].shift(periods=1, freq = day_us) # shifted by 1 day

The problem is that my DatetimeIndex is not equal to USFederalHolidayCalendar(). Can someone please tell me how I can use pd.offsets.CustomBusinessMonthEnd (and also pd.offsets.CustomBusinessDay) with my own custom DatetimeIndex?

If not, has any of you an idea how to tackle this issue in a different way?

Thanks a lot for your help!

LisaBinder
  • 41
  • 5
  • did you try the code with the US holidays? I mean when you shift the column to next monthend, it does though an error, no? as you have several values to assign to the same date – Ben.T Feb 28 '22 at 21:46

0 Answers0