1

I am trying to recreate the work done in this video, CppDay20Interoperable AI: ONNX & ONNXRuntime in C++ (M. Arena, M.Verasani).The github repository for the demo code is here .So far I have trained a regression model using TensorFlow and have converted into ONNX for inference in c++. But the created ONNX runtime session is unable to read the input shape of my model; the input shape is returning a value of -1.

Ort::Env env;
Ort::Session session{env,model_path, Ort::SessionOptions{} };

Ort::AllocatorWithDefaultOptions allocator;
auto* inputName = session.GetInputName(0, allocator);
std::cout << "Input name: " << inputName << "\n";
auto* outputName = session.GetOutputName(0, allocator);
std::cout << "Output name: " << outputName << "\n";
auto inputShape = session.GetInputTypeInfo(0).GetTensorTypeAndShapeInfo().GetShape();
//model has 5 inputs
std::vector<float> inputValues = {1, 2, 3, 4, 5 }; 

// where to allocate the tensors
auto memoryInfo = Ort::MemoryInfo::CreateCpu(OrtDeviceAllocator, OrtMemTypeCPU);

// create the input tensor (this is not a deep copy!)
auto inputOnnxTensor = Ort::Value::CreateTensor<float>(memoryInfo, 
    inputValues.data(), inputValues.size(), 
    inputShape.data(), inputShape.size());
    
// the API needs the array of inputs you set and the array of outputs you get
array inputNames = { inputName };
array outputNames = { outputName };

// finally run the inference!
auto outputValues = session.Run(
    Ort::RunOptions{ nullptr }, // e.g. set a verbosity level only for this run
    inputNames.data(), &inputOnnxTensor, 1, // input to set
    outputNames.data(), 1);

output :

Number of model inputs: 1
Number of model outputs: 1
Input name: input_1
Output name: Identity
tried creating tensor with negative value in shape

Any suggestions to make the inference code work?

JINU RAJ
  • 63
  • 1
  • 7
  • What's the exact content in `inputShape`? Can you share a link to your model? – Lifu Huang Aug 23 '21 at 07:11
  • Its an ANN model trained in TF 2 and I have tried doing the same onnx coversion for a LSTM network too, but both of the models are showing same error https://gitlab.com/jinu_ml/lstm-predictions – JINU RAJ Aug 23 '21 at 10:56
  • Iam providing link to the model, rally_lin.onnx and rally_lstm.onnx are the two onnx models (linear regression and LSTM network) ** https://gitlab.com/jinu_ml/lstm-predictions/-/blob/main/rally_lin.onnx – JINU RAJ Aug 23 '21 at 11:03

1 Answers1

0

Your model takes an input tensor of shape <unk, 5>, with the first dimension being unknown (usually for dynamic batch size), so it is expected to see -1 at the first dimension when you call session.GetInputTypeInfo(0).GetTensorTypeAndShapeInfo().GetShape().

enter image description here

Before feeding actual data to your model, you need to set a concrete shape for your tensor, in your case, you should perhaps set the first dimension to 1 manually.

Lifu Huang
  • 11,930
  • 14
  • 55
  • 77
  • the problem is its an LSTM model and the none shape is dynamically assigned in tensorflow 2.0 and manually adding 1 as first dimension throws error . [Lstm_model.png](https://gitlab.com/jinu_ml/lstm-predictions/-/blob/main/model_lstm.png) – JINU RAJ Aug 30 '21 at 05:29