How insert data in an ONNX as float32[N,60,1] in ML.NET

Question

I'm using ML.NET and I want to insert as input a float32[N, 60, 1](as in the picture). I don't figure how to pass the data. I'm trying with this class:

public class OnnxInput
{
    [ColumnName("lstm_input")]
    public float lstm_input { get; set; }
}

var input = new OnnxInput[length][];

// Here I load the data into the input variable

var dataView = mlContext.Data.LoadFromEnumerable(input);
var pipeline = mlContext.Transforms.
            ApplyOnnxModel(
                    modelFile: modelLocation,
                    inputColumnNames: new[] { TinyYoloModelSettings.ModelInput },
                    outputColumnNames: new[] { TinyYoloModelSettings.ModelOutput }
                );
var model = pipeline.Fit(data);

creating this matrix, when I try to fit the data into the pipeline I have the error: System.ArgumentOutOfRangeException: 'Could not determine an IDataView type and registered custom types for member SyncRoot (Parameter 'rawType')'

Trying with another approach, with this input class:

public class OnnxInput
{
    [ColumnName("lstm_input")]
    public float[] lstm_input { get; set; }
}

var input = new OnnxInput[realLength];

// Here I load the data into the input variable

var dataView = mlContext.Data.LoadFromEnumerable(input);
var pipeline = mlContext.Transforms.
            ApplyOnnxModel(
                    modelFile: modelLocation,
                    inputColumnNames: new[] { TinyYoloModelSettings.ModelInput },
                    outputColumnNames: new[] { TinyYoloModelSettings.ModelOutput }
                );
var model = pipeline.Fit(data);

creating this matrix, when I try to fit the data into the pipeline I have the error: System.InvalidOperationException: 'Variable length input columns not supported'

score 6 · Accepted Answer · answered May 16 '21 at 17:19

6

The variable input error (Variable length input columns not supported) just means your model is expecting a fixed sized input. Specifically, you can add the attribute [VectorType(60, 1)] on top of the property lstm_input in OnnxInput class.

answered May 16 '21 at 17:19

Apidcloud

3,558
2
21
23

why float[N, 60, 1] is represented as just [VectorType(60, 1)]? How about N? And how to represent the output float32[ N, 1 ]? I'm stuck on this https://stackoverflow.com/questions/75022045/ml-net-how-to-define-shape-vectors-for-gpt2-onnx-model If you could help I would thank you! – Murilo Maciel Curti Jan 05 '23 at 17:18
1

because N is not known (think about it as the model being called N times to perform a prediction). 60 is the number of features the model needs to do a prediction. The output is a single value, right? Then it shouldn't need an attribute – Apidcloud Jan 09 '23 at 13:14

How insert data in an ONNX as float32[N,60,1] in ML.NET

1 Answers1