Cannot Run Azure ML Model Locally, Help on Entry script

Question

I am working on a project where a model will be created using Azure ML(ml.azure.com),

I plan to utilize Azure ML Designer eventually, So I picked up a sample pipeline (Regression - Automobile Price Prediction (Basic)). I then created a realtime inference pipeline and created a realtime endpoint.

Post this, I am able to fetch predictions from this model from a .Net client over internet as expected.

As a part of the same project, I eventually want to fully eliminate the internet dependency/Affiliated latency with Predictions. I understand this can be achieved by packaging the model and downloading a docker image that runs on a local computer and exposes the model via a scoring web service.

To package the model and download it as a docker image, I utilized the following python code, this would execute on my local PC:

import azureml.core
print(azureml.core.VERSION)


from azureml.core import Workspace
ws = Workspace.get(name="TestMLWorkSpace", subscription_id='XXXXXX-a9c4-4426-XXXX-XXXXXXX', resource_group='MyResourceGroup')

from azureml.core.experiment import Experiment
experiment = Experiment(workspace=ws, name='TestAutomobileSampleExp')

list_runs = experiment.get_runs()
for run in list_runs:
    print(run.id)

list_experiments = Experiment.list(ws)


from azureml.core.model import Model
import os

model = Model(workspace=ws, name="amlstudio-regrealtimeendpoint")
model.download(target_dir=os.getcwd(), exist_ok=1 )

from azureml.core import Environment
from azureml.core.model import InferenceConfig

env = Environment.get(workspace=ws, name="AzureML-Minimal")
inference_config = InferenceConfig(
    environment=env,
    source_directory="./",
    entry_script="./score.py",
)



package = Model.package(ws, [model], inference_config)
package.wait_for_creation(show_output=True)

package.pull()
#checked image id using --->   docker images
# replace image id from the result above in the following command
# docker run -p 6789:5001 --name mycontainer c262eaa7bab9

#  http://localhost:6789/score

Now when I run this script, a docker image is downloaded on my computer and available to be run using docker.

What is the problem?

As a part of creating the inference config, I need to specify an entry script called score.py It seems I cannot run the docker image because the entry script is likely not correct.

Here is how my score.py currently looks like:

import os import json

from azureml.core.model import Model import logging

def init():
    global model
    logging.basicConfig(level=logging.DEBUG)
    model_path = Model.get_model_path(model_name='amlstudio-regrealtimeendpoint')
    print(Model.get_model_path(model_name='amlstudio-regrealtimeendpoint'))
    #How do I load the model here??


def run(input_data):
    try:
        data = json.loads(input_data)['data']
        data = np.array(data)
        result = model.predict(data)
        return json.dumps({"result": result.tolist()})
    except Exception as e:
        result = str(e)
        # return error message back to the client
        return json.dumps({"error": result})

I built this entry script from Microsoft materials but it does not work and Docker image fails to initiate.

My question is,

Is my source Dir in inference config correct? what should be my source Directory given the fact that I just want the model to run locally be able to provide predictions locally?
what changes should I make to this entry script to get the scoring to work locally on my computer?

Thanks,

Ram · Answer 1 · 2021-07-22T01:51:47.430

Troubleshooting Doc: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-troubleshoot-prebuilt-docker-image-inference

If you want to build a Docker image that encapsulates your model and its dependencies, you can use the model packaging option. The output image will be pushed to your workspace's ACR.

You must include an Environment object in your inference configuration to use Model.package().

package = Model.package(ws, [model], inference_config) package.wait_for_creation(show_output=True) # Or show_output=False to hide the Docker build logs. package.pull() Instead of a fully-built image, you can also generate a Dockerfile and download all the assets needed to build an image on top of your Environment.

package = Model.package(ws, [model], inference_config, generate_dockerfile=True) package.wait_for_creation(show_output=True) package.save("./local_context_dir")

Cannot Run Azure ML Model Locally, Help on Entry script

1 Answers1