How to calculate SMAPE for groups in time-series data in python?

Question

My data looks like following, and I am using facebook FbProphet for prediction. Next I would like to calculate SMAPE for each group in my dataframe. I found the function described by Kaggle user here But I am not sure How to implement in my current code. So that SMAPE can calculate for each group. Additionally, I know that fbProphet has validation function, but I would like to calculate SMAPE for each group.

Note: I am new to python, provide explanation with code.

Dataset

import pandas as pd
data = {'Date':['2017-01-01', '2017-01-01','2017-01-01','2017-01-01','2017-01-01','2017-01-01','2017-01-01','2017-01-01',
               '2017-02-01', '2017-02-01','2017-02-01','2017-02-01','2017-02-01','2017-02-01','2017-02-01','2017-02-01'],'Group':['A','A','B','B','C','C','D','D','A','A','B','B','C','C','D','D'],
       'Amount':['12.1','13.2','15.1','10.7','12.9','9.0','5.6','6.7','4.3','2.3','4.0','5.6','7.8','2.3','5.6','8.9']}
df = pd.DataFrame(data)
print (df)

Code so far...

def get_prediction(df):
    prediction = {}
    df = df.rename(columns={'Date': 'ds','Amount': 'y', 'Group': 'group'})
    df=df.groupby(['ds','group'])['y'].sum()
    df=pd.DataFrame(df).reset_index()
    list_articles = df.group.unique()

    for group in list_articles:
        article_df = df.loc[df['group'] == group]
        # set the uncertainty interval to 95% (the Prophet default is 80%)
        my_model = Prophet(weekly_seasonality= True, daily_seasonality=True,seasonality_prior_scale=1.0)
        my_model.fit(article_df)
        future_dates = my_model.make_future_dataframe(periods=6, freq='MS')
        forecast = my_model.predict(future_dates)
        prediction[group] = forecast
        my_model.plot(forecast)
    return prediction

This might help https://stats.stackexchange.com/questions/145490/minimizing-symmetric-mean-absolute-percentage-error-smape And are you trying to calculate smape on this group ```for group in list_articles:``` — jayprakashstar, May 03 '19 at 06:38
@jayprakashstar..yes..I am trying to calculate smape for each group located in my "group" column and predicted values obtained using prophet — biggboss2019, May 03 '19 at 21:11

tvgriek · Accepted Answer · 2019-05-08T08:58:14.053

You can still use fbprophet's own cross_validation function but use your own scoring. Here is a nice blog from uber on how they do backtesting (sliding window & expanding window): https://eng.uber.com/forecasting-introduction/

fbprophet's cv function operates on a sliding window. If that is ok for you can use this in combination with a custom scoring function. I think a nice way is to extend Prophet and implement a .score() method.

Here an example implementation:

from fbprophet import Prophet
from fbprophet.diagnostics import cross_validation
import numpy as np

class ProphetEstimator(Prophet):
    """
        Wrapper with custom scoring
    """

    def __init__(self, *args, **kwargs):
        super(ProphetEstimator, self).__init__(*args, **kwargs)

    def score(self):
        # cross val score reusing prophets own cv implementation
        df_cv = cross_validation(self, horizon='6 days')
        # Here decide how you want to calculate SMAPE.
        # Here each sliding window is summed up, 
        # and the SMAPE is calculated over the sum of periods, for all windows.
        df_cv = df_cv.groupby('cutoff').agg({
            "yhat": "sum",
            'y': "sum"
        })
        smape = self.calc_smape(df_cv['yhat'], df_cv['y'])
        return smape

    def calc_smape(self, y_hat, y):
        return 100/len(y) * np.sum(2 * np.abs(y_hat - y) / (np.abs(y) + np.abs(y_hat)))


def get_prediction(df):
    prediction = {}
    df = df.rename(columns={'Date': 'ds','Amount': 'y', 'Group': 'group'})
    df=df.groupby(['ds','group'])['y'].sum()
    df=pd.DataFrame(df).reset_index()
    list_articles = df.group.unique()

    for group in list_articles:
        article_df = df.loc[df['group'] == group]
        # set the uncertainty interval to 95% (the Prophet default is 80%)
        my_model = ProphetEstimator(weekly_seasonality= True, daily_seasonality=True,seasonality_prior_scale=1.0)
        my_model.fit(article_df)
        smape = my_model.score() # store this somewhere
        future_dates = my_model.make_future_dataframe(periods=6, freq='MS')
        forecast = my_model.predict(future_dates)
        prediction[group] = (forecast, smape)
        my_model.plot(forecast)
    return prediction

How to calculate SMAPE for groups in time-series data in python?

1 Answers1