import local module in azureml endpoint

Question

I am trying to create and endpoint for my azureml model. All the examples that I have seen for the moment use models from the library sklearn, so no problem to load the model from .pkl file.

In my case, my model is of a custom class that I wrote my self "bert_based_model".

The problem now is when compiling my score.py file, I get an error :

ModuleNotFoundError: No module named 'bert_based_model'

How can I import my custom model in azureml endpoint?

Thank you.

score 0 · Accepted Answer · answered Oct 16 '22 at 04:11

Assuming you have a script my_module.py that contains your custom model class MyBertModel, you should place the script under the source_directory of the endpoint and import the class in the entry script.

So, if this is your inference configuration:

inference_config = InferenceConfig(
    environment=env,
    source_directory='./endpoint_source',
    entry_script="./score.py",
)

This is how ./endpoint_source/ should look like:

.
└── endpoint_source
    ├── score.py
    ├── my_module.py
    └── ...

And this is how your entry script score.py should look like:

from my_module import MyBertModel

# other imports
...

def init():
    global model
    
    # load the model here, from pickle file or using the custom class
    model = ...
   

def run(data):
    # use the model here to predict on data
    ...

import local module in azureml endpoint

1 Answers1