AzureML - How to create a web service from a clustering model

Question

I've created an Inference Pipeline that clusters a set of data using K-means. When run from the designer it works fine and returns cluster assignments for the input data. Once deployed as a web service, the service always returns an empty set of results. Has anybody managed to do something similar?

Real-time Endpoint test - 0 items returned

K-means Clustering Model Realtime Inference Pipeline

Web Service Output Results_dataset

UPDATE: 4/22

In the Deployment logs I see this Warning

The message depends on whether I'm using an AksCompute instance in Production or Dev-Test mode.

In Dev-Test mode the warning is: output_name WebServiceOutput0 exists in schema but not in data

In Production mode I get: output_name output1 exists in schema but not in data

I've gone around in circles and still can't get anything in my returned dataset. Also logged a support ticket with MS so if they help me I'll post the answer here.

UPDATE: 4/24

Deployment log when testing endpoint:

    Init: Graph has been loaded
    2023-04-24 17:32:43,725 | root | INFO | Users's init has completed successfully
2023-04-24 17:32:43,726 | root | INFO | Skipping middleware: dbg_model_info as it's not enabled.
2023-04-24 17:32:43,726 | root | INFO | Skipping middleware: dbg_resource_usage as it's not enabled.
Found swagger file: /var/azureml-app/swagger.json
Swagger file loaded.
2023-04-24 17:32:43,733 | root | INFO | Scoring timeout is found from os.environ: 60000 ms
2023-04-24 17:32:50,235 | root | INFO | 200
127.0.0.1 - - [24/Apr/2023:17:32:50 +0000] "GET /swagger.json HTTP/1.0" 200 3301 "-" "-"
2023-04-24 17:32:53,910 | root | INFO | Scoring Timer is set to 60.0 seconds
Handling http request - Start:
2023-04-24 17:32:53,911 studio.azureml.designer.serving.dagengine.request_handler INFO       |   Run: is_classic = False, with_details = False, verbose = True
2023-04-24 17:32:53,911 studio.core          INFO       |   Pre-processing - Start:
2023-04-24 17:32:53,911 studio.core          INFO       |   Pre-processing - End with 0.0001s elapsed.
2023-04-24 17:32:53,912 studio.core          INFO       |   Processing - Start:
2023-04-24 17:32:53,932 studio.core          INFO       |   Processing - End with 0.0207s elapsed.
2023-04-24 17:32:53,933 studio.core          INFO       |   Post-processing - Start:
2023-04-24 17:32:53,933 studio.azureml.designer.serving.dagengine.processor WARNING    |   |   output_name WebServiceOutput0 exists in schema but not in data
2023-04-24 17:32:53,934 studio.core          INFO       |   Post-processing - End with 0.0006s elapsed.
2023-04-24 17:32:53,934 studio.core          INFO       Handling http request - End with 0.0235s elapsed.
2023-04-24 17:32:53,940 studio.azureml.designer.serving.dagengine.request_handler DEBUG      Run: output data(raw) = {"Results": {}}
2023-04-24 17:32:53,941 | root | INFO | run() output is HTTP Response
2023-04-24 17:32:53,941 | root | INFO | 200
127.0.0.1 - - [24/Apr/2023:17:32:53 +0000] "POST /score?verbose=true HTTP/1.0" 200 15 "-" "-"
2023-04-24 17:33:04,283 | root | INFO | 200
127.0.0.1 - - [24/Apr/2023:17:33:04 +0000] "GET /swagger.json HTTP/1.0" 200 3301 "-" "-"
2023-04-24 17:33:11,437 | root | INFO | 200
127.0.0.1 - - [24/Apr/2023:17:33:11 +0000] "GET /swagger.json HTTP/1.0" 200 3301 "-" "-"
2023-04-24 17:33:12,932 | root | INFO | Scoring Timer is set to 60.0 seconds
2023-04-24 17:33:12,932 studio.core          INFO       Handling http request - Start:
2023-04-24 17:33:12,932 studio.azureml.designer.serving.dagengine.request_handler INFO       |   Run: is_classic = False, with_details = False, verbose = True
2023-04-24 17:33:12,932 studio.core          INFO       |   Pre-processing - Start:
2023-04-24 17:33:12,933 studio.core          INFO       |   Pre-processing - End with 0.0001s elapsed.
2023-04-24 17:33:12,933 studio.core          INFO       |   Processing - Start:
2023-04-24 17:33:12,949 studio.core          INFO       |   Processing - End with 0.0163s elapsed.
2023-04-24 17:33:12,949 studio.core          INFO       |   Post-processing - Start:
2023-04-24 17:33:12,949 studio.azureml.designer.serving.dagengine.processor WARNING    |   |   output_name WebServiceOutput0 exists in schema but not in data
2023-04-24 17:33:12,949 studio.core          INFO       |   Post-processing - End with 0.0001s elapsed.
2023-04-24 17:33:12,949 studio.core          INFO       Handling http request - End with 0.0170s elapsed.
2023-04-24 17:33:12,949 studio.azureml.designer.serving.dagengine.request_handler DEBUG      Run: output data(raw) = {"Results": {}}
2023-04-24 17:33:12,950 | root | INFO | run() output is HTTP Response
2023-04-24 17:33:12,950 | root | INFO | 200
127.0.0.1 - - [24/Apr/2023:17:33:12 +0000] "POST /score?verbose=true HTTP/1.0" 200 15 "-" "-"
2023-04-24 17:42:24,397 | root | INFO | 200
127.0.0.1 - - [24/Apr/2023:17:42:24 +0000] "GET /swagger.json HTTP/1.0" 200 3301 "-" "-"

**DO NOT post images of code, data, error messages, etc.** - copy or type the text into the question. [ask] — Rob, Apr 22 '23 at 16:40

score 0 · Answer 1 · answered Apr 21 '23 at 05:29

While deploying to real-time endpoint there is two option available in compute type.

1.AksCompute

2.Azure Container instance

Like below

enter image description here

I tried deploying the real time inference pipeline. in both options. It worked successfully. This is the results.

enter image description here

This endpoint is using Azure container instance and

enter image description here

this is another end point using AksCompute. And this my web service output result.

enter image description here

and realtime Inference pipeline.

enter image description here

Try creating endpoint using both compute options.

To create using AksCompute follow below steps.

First, you need to create kubernates cluster. Goto Compute in Ml studio workspace and click on New

enter image description here

Then select AksCompute option. In the next page select location and VM.

enter image description here

Click Next to Advanced settings.

enter image description here

Give a compute name and select Dev-test if your using for development purpose. Then click on Create. It takes some time to create.

Now let's deploy the real-time inference pipeline. After submitting the inference pipeline, the job starts running and you will get an web service output and options as below.

enter image description here

Here you can find Deploy option, click on it.

enter image description here

Next, you will get an option as above. Select Deploy new real-time endpoint. Give a name to end point, then select AksCompute as compute type and select compute name from drop down which your created before in Kubernetes cluster. Then click on Deploy, the deployment will start which takes some time to deploy.

After deployment test the result in endpoint Test tab or else you use Postman using endpoint URL and Authentication Key

enter image description here This deployment is using AksCompute and try deploying using Azure container service. While deploying just select container service as compute type and click on deploy.

If you still getting empty result, go to deployment logs in endpoint there you can find the logs for the pipeline run and result like below. There you can see what went wrong.

2023-04-20 19:18:08,363 studio.module        INFO       |   |   |   |   Select column indexes from Dataset
2023-04-20 19:18:08,365 studio.core          DEBUG      |   |   |   |   return:
2023-04-20 19:18:08,365 studio.core          DEBUG      |   |   |   |   |   [0] = <DataTable (3 Rows, 1 Cols) at 0x00007FD783962430>
2023-04-20 19:18:08,365 studio.core          INFO       |   |   |   SelectColumnsModule.run - End with 0.0085s elapsed.
2023-04-20 19:18:08,365 studio.core          INFO       |   |   Executing node 4: Select Columns in Dataset - End with 0.0116s elapsed.
2023-04-20 19:18:08,373 studio.core          INFO       |   Processing - End with 0.1412s elapsed.
2023-04-20 19:18:08,374 studio.core          INFO       |   Post-processing - Start:
2023-04-20 19:18:08,374 studio.core          INFO       |   Post-processing - End with 0.0000s elapsed.
2023-04-20 19:18:08,374 studio.core          INFO      
Handling http request - End with 0.1420s elapsed.
2023-04-20 19:18:08,374 studio.azureml.designer.serving.dagengine.request_handler DEBUG      Run: output data(raw) = {"Results": {"WebServiceOutput0": [{"Assignments": 1}, {"Assignments": 0}, {"Assignments": 2}]}}
2023-04-20 19:18:08,374 | root | INFO | run() output is HTTP Response
2023-04-20 19:18:08,374 | root | INFO | 200
127.0.0.1 - - [20/Apr/2023:19:18:08 +0000] "POST /score?verbose=true HTTP/1.0" 200 96 "-" "-"

Thanks for such a thorough answer JayashankarGS! I tried with both compute types and even created a new AksCompute instance configured for Production rather than Dev-test. Everything I try always gives me zero results when I test the endpoint, despite the Inference Pipeline showing 3 rows when submitted in the designer. When I look at the deployment logs I see a warning message which I think could be the root of this: WARNING || output_name WebServiceOutput0 exists in schema but not in data — Simon, Apr 22 '23 at 16:20
Are you able to see **Assign Data to Clusters** and **Select Columns in Dataset** module runs in deployment logs, that is node 3 and 4. Because i can see the pipeline runs upto node 2 (**Apply Transformation**) in your deployment logs. Please check your pipeline at both **Extract N-gram Features from Text** nodes. — JayashankarGS, Apr 24 '23 at 07:25
Weirdly none of the nodes seem to be running according to the deployment logs however there's no errors and just the warning about WebServiceOutput0 not existing in schema. I just tried recreating the training and inference pipelines from scratch but still get the same result. I even removed the Apply Transformation node and still nothing. — Simon, Apr 24 '23 at 17:50
Just created a brand new workspace to see if that would clear this issue but that didn't work either. — Simon, Apr 24 '23 at 21:15

AzureML - How to create a web service from a clustering model

1 Answers1

Linked