Pandas diff-function: NotImplementedError

Asked Aug 23 '22 at 21:38

Active Aug 24 '22 at 10:16

Viewed 64 times

When I use the diff function in my snippet:

for customer_id, cus in tqdm(df.groupby(['customer_ID'])):
        # Get differences
        diff_df1 = cus[num_features].diff(1, axis = 0).iloc[[-1]].values.astype(np.float32)

I get:

NotImplementedError

The exact same code did run without any error before (on Colab), whereas now I'm using an Azure DSVM via JupyterHub and I get this error.

I already found this pandas pd.DataFrame.diff(axis=1) NotImplementationError but the solution doesnt work for me as I dont have any Date types. Also I did upgrade pandas but it didnt change anything.

EDIT:
I have found that the error occurs when the datatype is 'int16' or 'int8'. Converting the dtypes to 'int64' solves it. However I leave the question open in case someone can explain it or show a solution that works with int8/int16.

edited Aug 24 '22 at 10:16

asked Aug 23 '22 at 21:38

paka

What is `num_features`? – BeRT2me Aug 23 '22 at 22:52
any changes in the data? for example `num_features` or `customer_ID`? – blackraven Aug 23 '22 at 22:56
@BeRT2me a list with column names (numerical columns), please see my Edit – paka Aug 24 '22 at 10:17
@perpetualstudent No, please see Edit – paka Aug 24 '22 at 10:18
oh i see, it could be due to the data type difference between python native int type and the numpy int types – blackraven Aug 24 '22 at 10:55

Pandas diff-function: NotImplementedError

0 Answers0