Various errors using VisionServiceClient in XamarinForms

Question

I am trying to create a simple Xamarin forms app which allows the user to browse for or take a photo and have azure cognitive services tag the photo using a custom vision model.

I am unable to get the client to successfully authenticate or find a resource per the error message in the exception produced by the VisionServiceClient. Am I missing something? What would be the correct values to use for the arguments to VisionServiceClient?

All keys have been removed from the below images, they are populated.

Exception thrown in VS2017:

'Microsoft.ProjectOxford.Vision.ClientException' in System.Private.CoreLib.dll

Call to VisionServiceClient:

private const string endpoint = @"https://eastus2.api.cognitive.microsoft.com/vision/prediction/v1.0";
private const string key = "";

VisionServiceClient visionClient = new VisionServiceClient(key, endpoint);
VisualFeature[] features = { VisualFeature.Tags, VisualFeature.Categories, VisualFeature.Description };
try
{
     AnalysisResult temp = await visionClient.AnalyzeImageAsync(imageStream,                    
     features.ToList(), null);

     return temp;
}
catch(Exception ex)
{
     return null;
}

VS Exception Error:

Azure Portal for cognitive services:

Custom Vision Portal:

cthrash · Accepted Answer · 2019-01-29T20:30:16.867

1

It looks like you're confusing the Computer Vision and the Custom Vision APIs. You are attempting to use the client SDK for the former using the API key of the latter.

For .NET languages, you'll want the Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction NuGet package.

Your code will end up looking something like this:

ICustomVisionPredictionClient client = new CustomVisionPredictionClient()
{
    ApiKey = PredictionKey,
    Endpoint = "https://southcentralus.api.cognitive.microsoft.com"
};
ImagePrediction prediction = await client.PredictImageAsync(ProjectId, stream, IterationId);

edited Jan 29 '19 at 20:30

answered Jan 29 '19 at 18:49

cthrash

2,938
2
11
10

You are correct cthrash, where do you get the Guid for the project id? You can get it as a string off the portal so do I just do a Guid.Parse to pass the I'd to PredictImageAsync? – Jan 29 '19 at 19:11
On the same Project Settings page where you see your Prediction Key, you should also see a Project Id. The SDK expects a string, no need to parse. – cthrash Jan 29 '19 at 19:16
The call to PredictImageAsync expects a Guid type as the first parameter for project id not a string, that's why I'm confused. – Jan 29 '19 at 19:20
My mistake - you are correct, you'll need to `new Guid('...')` or use of the `Parse` methods. – cthrash Jan 29 '19 at 19:23
When I try that I now get an http error for the rest request. The message is "Operation returned an invalid status code 'NotFound'" – Jan 29 '19 at 19:29
Do you see an iteration name 'Workspace' on your project page? – cthrash Jan 29 '19 at 19:43
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/187514/discussion-between-cthrash-and-schwagmister). – cthrash Jan 29 '19 at 19:45
Updated the answer - you'll want to supply the Iteration ID (a GUID) unless you've set a default one. – cthrash Jan 29 '19 at 20:30

score 0 · Answer 2 · answered Jan 30 '19 at 15:55

Thank you to cthrash for the extended help and talking with me in chat. Using his post along with a little troubleshooting I have figured out what works for me. The code is super clunky but it was just to test and make sure I'm able to do this. To answer the question:

Nuget packages and classes

Using cthrash's post I was able to get both the training and prediction nuget packages installed, which are the correct packages for this particular application. I needed the following classes:

Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction.Models Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.Models

Endpoint Root

Following some of the steps Here I determined that the endpoint URL's only need to be the root, not the full URL provided in the Custom Vision Portal. For instance,

https://southcentralus.api.cognitive.microsoft.com/customvision/v2.0/Prediction/

Was changed to

https://southcentralus.api.cognitive.microsoft.com

I used both the key and endpoint from the Custom Vision Portal and making that change I was able to use both a training and prediction client to pull the projects and iterations.

Getting Project Id

In order to use CustomVisionPredictionClient.PredictImageAsync you need a Guid for the project id and an iteration id if a default iteration is not set in the portal.

I tested two ways to get the project id,

Using project id string from portal

Grab the project id string from the portal under the project settings.
For the first argument to PredictImageAsync pass

Guid.Parse(projectId)

Using the training client

Create a new CustomVisionTrainingClient
To get a list of <Project> use

TrainingClient.GetProjects().ToList()
In my case I only had a single project so I would just need the first element.

Guid projectId = projects[0].Id

Getting Iteration Id

To get the iteration id of a project you need the CustomVisionTrainingClient.

Create the client
To get a list of <Iteration> use

client.GetIterations(projectId).ToList()
In my case I had only a single iteration so I just need the first element.

Guid iterationId = iterations[0].Id

I am now able to use my model to classify images. In the code below, fileStream is the image stream passed to the model.

public async Task<string> Predict(Stream fileStream)
{
    string projectId = "";
    //string trainingEndpoint = "https://southcentralus.api.cognitive.microsoft.com/customvision/v2.2/Training/";
    string trainingEndpoint = "https://southcentralus.api.cognitive.microsoft.com/";
    string trainingKey = "";
    //string predictionEndpoint = "https://southcentralus.api.cognitive.microsoft.com/customvision/v2.0/Prediction/";
    string predictionEndpoint = "https://southcentralus.api.cognitive.microsoft.com";
    string predictionKey = "";

    CustomVisionTrainingClient trainingClient = new CustomVisionTrainingClient
    {
        ApiKey = trainingKey,
        Endpoint = trainingEndpoint
    };

    List<Project> projects = new List<Project>();

    try
    {
        projects = trainingClient.GetProjects().ToList();
    }
    catch(Exception ex)
    {
        Debug.WriteLine("Unable to get projects:\n\n" + ex.Message);
        return "Unable to obtain projects.";
    }

    Guid ProjectId = Guid.Empty;

    if(projects.Count > 0)
    {
        ProjectId = projects[0].Id;
    }

    if (ProjectId == Guid.Empty)
    {
        Debug.WriteLine("Unable to obtain project ID");
        return "Unable to obtain project id.";
    }

    List<Iteration> iterations = new List<Iteration>();

    try
    {
        iterations = trainingClient.GetIterations(ProjectId).ToList();
    }
    catch(Exception ex)
    {
        Debug.WriteLine("Unable to obtain iterations.");
        return "Unable to obtain iterations.";
    }

    foreach(Iteration itr in iterations)
    {
        Debug.WriteLine(itr.Name + "\t" + itr.Id + "\n");
    }

    Guid iteration = Guid.Empty;

    if(iterations.Count > 0)
    {
        iteration = iterations[0].Id;
    }

    if(iteration == Guid.Empty)
    {
        Debug.WriteLine("Unable to obtain project iteration.");
        return "Unable to obtain project iteration";
    }

    CustomVisionPredictionClient predictionClient = new CustomVisionPredictionClient
    {
        ApiKey = predictionKey,
        Endpoint = predictionEndpoint
    };

    var result = await predictionClient.PredictImageAsync(Guid.Parse(projectId), fileStream, iteration);

    string resultStr = string.Empty;

    foreach(PredictionModel pred in result.Predictions)
    {
        if(pred.Probability >= 0.85)
            resultStr += pred.TagName + " ";
    }

    return resultStr;
}

I also discovered that the newest iteration is always the first element of the iteration list. So using list[0] or list.first() would always get the newest iteration. — , Feb 01 '19 at 14:54

Various errors using VisionServiceClient in XamarinForms

2 Answers2