0

I am building a web app and training a machine learning model with autoSklearn using dash plotly and in a multi page app, i encountered such a problem when the model is successfully trained, but at the same time I cannot save the trained model in pickle inside the callback and the following error pops up (_pickle.PicklingError: Can’t pickle <function error at 0x7f1ebbd3b820>: import of module ‘pages.dashboard.experiment’ failed).

after unsuccessfully saving to pickle, I tried to save the model to joblib but also unsuccessfully. I am getting the following error (Exception: dash.register_page() can’t be called within a callback as it updates dash.page_registry, which is a global variable)

This error happens in multipage application here is the minimal code

@callback(
   Output(...),
   Input(..),
 )
def example_def(_):
   ...
   cls = autosklearn.classification.AutoSklearnClassifier(
        time_left_for_this_task=120,
        per_run_time_limit=30,
        scoring_functions=[roc_auc],
    )
    cls.fit(X_train, y_train)

    pickle.dump(cls, open('AUTOSKLEARN.pkl', "wb"))
    
    #with open('classifier.pkl', 'wb') as f:
    #       pickle.dump(cls, f)

    # joblib.dump(cls, 'final_model.joblib')

i have published the code in github where you can run all the code and reproduce the error

i don't get this error in single page app, probably a problem with dash.pages but couldn't fix the error and stuck on it for days, any advice would be greatly appreciated

molbdnilo
  • 64,751
  • 3
  • 43
  • 82
Sohibjon
  • 41
  • 5

1 Answers1

0

I think I am working on a similar use case. I choose a different strategy.

I use the dash "Store" to the model within the app. I also push the labels and model parameters to a DB table. I have a chron job that retrains the model and then pushes to production.

I am trying to improve speed now.

sabrooks
  • 11
  • 1