I am evaluating customvision.ai for training image classification model and then downloading that model as an onnx file that would be consumed within a .Net Windows forms app.
I created a new project, uploaded few images, tagged them and was able to fetch predictions from the model within Customvision.ai .The model accuracy was acceptable. CustomVision allows you to download a model as an ONNX file which can be deployed within a cross platform application. In my case I plan to deploy and consume the model within a Windows forms application.
When I download the model as onnx, I receive a zip file that contains the .onnx file and few others.
One of the file is Metadata_properties.json, and it has the following contents:
{
"CustomVision.Metadata.AdditionalModelInfo": "",
"CustomVision.Metadata.Version": "1.2",
"CustomVision.Postprocess.Method": "ClassificationMultiClass",
"CustomVision.Postprocess.Yolo.Biases": "[]",
"CustomVision.Postprocess.Yolo.NmsThreshold": "0.0",
"CustomVision.Preprocess.CropHeight": "0",
"CustomVision.Preprocess.CropMethod": "FullImageShorterSide",
"CustomVision.Preprocess.CropWidth": "0",
"CustomVision.Preprocess.MaxDimension": "0",
"CustomVision.Preprocess.MaxScale": "0.0",
"CustomVision.Preprocess.MinDimension": "0",
"CustomVision.Preprocess.MinScale": "0.0",
"CustomVision.Preprocess.NormalizeMean": "[0.0, 0.0, 0.0]",
"CustomVision.Preprocess.NormalizeStd": "[1.0, 1.0, 1.0]",
"CustomVision.Preprocess.ResizeMethod": "Stretch",
"CustomVision.Preprocess.TargetHeight": "300",
"CustomVision.Preprocess.TargetWidth": "300",
"Image.BitmapPixelFormat": "Rgb8",
"Image.ColorSpaceGamma": "SRGB",
"Image.NominalPixelRange": "Normalized_0_1"
}
What I understand from this file is that the eventual Tensor that would be provided to the model for inference would need to be stretch resized to 300x300, Normalized between 0 and 1, Mean set to zero and stdev set to 1. In order to consume this model within my code, here is what I put together from various online sources:
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;
using SixLabors.ImageSharp.Processing;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
//using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using Microsoft.ML.OnnxRuntime.Tensors;
using Microsoft.ML.OnnxRuntime;
using System.IO;
namespace TestONNXRunner
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
RunModel();
}
public void RunModel()
{
// Read paths
string modelFilePath = @"C:\ImageMLProjects\MarbleImagesDataset\OnnxModel\onnxdataset\model.onnx";
var LabelsDict = GetLabelMap(@"C:\ImageMLProjects\MarbleImagesDataset\OnnxModel\onnxdataset\labels.txt");
string imageFilePath = @"";
OpenFileDialog openFileDialog1 = new OpenFileDialog
{
InitialDirectory = @"C:\",
Title = "Browse Image Files",
CheckFileExists = true,
CheckPathExists = true,
FilterIndex = 2,
RestoreDirectory = true,
ReadOnlyChecked = true,
ShowReadOnly = true
};
if (openFileDialog1.ShowDialog() == DialogResult.OK)
{
imageFilePath = openFileDialog1.FileName;
// Read image
using Image<Rgb24> image = Image.Load<Rgb24>(imageFilePath);
// Resize image
image.Mutate(x =>
{
x.Resize(new ResizeOptions
{
Size = new SixLabors.ImageSharp.Size(300, 300),
Mode = ResizeMode.Stretch
});
});
// Preprocess image
Tensor<float> input = new DenseTensor<float>(new[] { 1, 3, image.Height, image.Width });
var mean = new[] { 0f, 0f, 0f };
var stddev = new[] { 1f, 1f, 1f };
for (int y = 0; y < image.Height; y++)
{
Span<Rgb24> pixelSpan = image.GetPixelRowSpan(y);
for (int x = 0; x < image.Width; x++)
{
input[0, 0, x, y] = ((pixelSpan[x].R / 255f) - mean[0]) / stddev[0];
input[0, 1, x, y] = ((pixelSpan[x].G / 255f) - mean[1]) / stddev[1];
input[0, 2, x, y] = ((pixelSpan[x].B / 255f) - mean[2]) / stddev[2];
}
}
// Setup inputs
var inputs = new List<NamedOnnxValue>
{
NamedOnnxValue.CreateFromTensor("data", input)
};
// Run inference
//int gpuDeviceId = 0; // The GPU device ID to execute on
//var session = new InferenceSession("model.onnx", SessionOptions.MakeSessionOptionWithCudaProvider(gpuDeviceId));
using var session = new InferenceSession(modelFilePath);
using IDisposableReadOnlyCollection<DisposableNamedOnnxValue> results = session.Run(inputs);
// Postprocess to get softmax vector
IEnumerable<float> output = results.First().AsEnumerable<float>();
float sum = output.Sum(x => (float)Math.Exp(x));
IEnumerable<float> softmax = output.Select(x => (float)Math.Exp(x) / sum);
// Extract top 10 predicted classes
IEnumerable<Prediction> top10 = softmax.Select((x, i) => new Prediction { Label = LabelsDict[i], Confidence = x })
.OrderByDescending(x => x.Confidence)
.Take(10);
// Print results to console
Console.WriteLine("Top 10 predictions for ResNet50 v2...");
Console.WriteLine("--------------------------------------------------------------");
foreach (var t in top10)
{
Console.WriteLine($"Label: {t.Label}, Confidence: {t.Confidence}");
}
}
}
public Dictionary<int, string> GetLabelMap(string LabelMapFile)
{
Dictionary<int, string> labelsDict = new Dictionary<int, string>();
if(File.Exists(LabelMapFile))
{
string data = File.ReadAllText(LabelMapFile);
string[] labels = data.Split('\n');
int i = 0;
foreach (var label in labels)
{
labelsDict.Add(i, label);
i++;
}
}
return labelsDict;
}
internal class Prediction
{
public string Label { get; set; }
public float Confidence { get; set; }
}
}
}
Now what is the problem?
I see no errors, Irrespective of what image I use for inference, I just get the same result.
Questions
- Should I structure the tensor differently? I am not sure if this is something to do with the way the Tensor is structured.
- The last updates to Customvision pages on Github was several years ago, Is CustomVision recommended for production usage in 2021? Should I be looking out for something else? The idea is to be able to build/train high quality image classification models with a low/zero code approach and then deploy the model onto on premise computers for use in low latency applications.
Any help in this regard would be appreciated
