2

I trained a Resnet50 (640x640) model from tensorflow models zoo and then converted it to "onnx" format and I am trying to run it with ML.Net

The object returned by the "PredictionEngine.Predict" is empty, all the fields are null.

The tutorial from Microsoft for object detection with ML.net is for the Yolo model, which doesn't seems to apply at all here since I had to change a lot of things to get to that point. Here's what I have so far :

var data = mlContext.Data.LoadFromEnumerable(new List<ImageNetData>());

// Define scoring pipeline
var pipeline = mlContext.Transforms.LoadImages(outputColumnName: "input_tensor", imageFolder: "", inputColumnName: nameof(ImageNetData.ImagePath))
    .Append(mlContext.Transforms.ResizeImages(outputColumnName: "input_tensor", imageWidth: ImageNetSettings.imageWidth, imageHeight: ImageNetSettings.imageHeight, inputColumnName: "input_tensor"))
    .Append(mlContext.Transforms.ExtractPixels(outputColumnName: "input_tensor", outputAsFloatArray: false))
    .Append(mlContext.Transforms.ApplyOnnxModel(modelFile: modelLocation, 
         outputColumnNames: new[] { "detection_boxes", "num_detections", "detection_classes", "detection_scores" }, 
         inputColumnNames: new[] { "input_tensor" }));

// Fit scoring pipeline
var model = pipeline.Fit(data);

var pred = mlContext.Model.CreatePredictionEngine<ImageNetData, ImageNetPrediction>(model);

var prediction = pred.Predict(new ImageNetData()
{
    ImagePath = @"usually_contains_full_path_to_img_here.png"
});
// here `prediction` is not null, but all the fields inside it are... <-----------------------

ImageNetData and ImageNetPrediction as follow:

public class ImageNetData
{
    [LoadColumn(0)]
    public string ImagePath;

    public static IEnumerable<ImageNetData> ReadFromFile(string imageFolder)
    {
        return Directory
            .GetFiles(imageFolder)
            .Where(filePath => Path.GetExtension(filePath) != ".md")
            .Select(filePath => new ImageNetData { ImagePath = filePath });
    }
}
public class ImageNetPrediction
{
    [ColumnName("detection_boxes:0")]
    public float[] DetectionBoxes;
}

I tried setting the ColumnName to detection_boxes ( without the :0 ) but it gives me this error when doing the Predict: Length of memory (1228800) must match product of dimensions (3).

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
pascx64
  • 904
  • 16
  • 31

0 Answers0