0

I'm trying to use ML.Net to consume an ONNX GPT-2 Model. https://github.com/onnx/models/blob/main/text/machine_comprehension/gpt-2/README.md

I'm stuck on defining the shape dicionary.

The following are the Input and Output model properties, extracted with Netron:

name: input1
type: int64[input1_dynamic_axes_1,input1_dynamic_axes_2,input1_dynamic_axes_3]

name: output1
type: float32[input1_dynamic_axes_1,input1_dynamic_axes_2,input1_dynamic_axes_3,50257]

name: output2
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output3
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output4
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output5
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output6
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output7
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output8
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output9
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output10
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output11
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output12
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

name: output13
type: float32[2,input1_dynamic_axes_2,12,input1_dynamic_axes_3,64]

What are the correct values for the shape dictionary? How to represente the input?

int64[ input1_dynamic_axes_1, input1_dynamic_axes_2, input1_dynamic_axes_3 ]

I've tryied:

{  "input1", new[] { 1, 32, 32 } }

Totally guessing...

And the next?

float32[ input1_dynamic_axes_1, input1_dynamic_axes_2, input1_dynamic_axes_3, 50257 ]

float32[ 2, input1_dynamic_axes_2, 12, input1_dynamic_axes_3, 64]

I'm passing Shape Dictionary to ApplyOnnxModel as in this article https://rubikscode.net/2021/10/25/using-huggingface-transformers-with-ml-net/

    var pipeline = _mlContext.Transforms
                    .ApplyOnnxModel(modelFile: bertModelPath,
                    shapeDictionary: shapeDictionary,
                                    outputColumnNames: outputColumnNames,
                                    inputColumnNames: inputColumnNames,
                                    gpuDeviceId: useGpu ? 0 : (int?)null,
                                    fallbackToCpu: true);

Thank you!

Murilo Maciel Curti
  • 2,677
  • 1
  • 21
  • 26

0 Answers0