Stack Overflow
Stack Overflow

Questions tagged [onnxruntime]

ONNX Runtime is a cross-platform inference and training machine-learning accelerator.

See onnxruntime github project.

292 questions
0
votes
2 answers

Pytorch to ONNX: Could not find an implementation for RandomNormalLike

I am trying to convert a fairly complex model from pytorch into ONNX. The conversion succeeds without error, but I am encountering this error when loading the model: Traceback (most recent call last): File "/home/***/***/***.py", line 50, in…
LemmeTestThat
  • 498
  • 7
  • 17
0
votes
0 answers

C++ OnnxRuntime_GPU: Session Run throws an access violation exception

I am writing a .dll extension that takes in NumPy images from Python and performs inference on them. The inference works fine on a CPU session. I then used the CUDA provider in hopes of getting a speedup, using the default settings. Ort::Session…
disguisedtoast
  • 149
  • 1
  • 4
  • 15
0
votes
1 answer

transform DenseTensor in Microsoft.ML.OnnxRuntime

I am currently trying to port the code from https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/blob/master/nnhash.py to C# using Micrisoft.ML.OnnxRuntime. For Image loading I use SixLabors.ImageSharp. So far loading the onnx session file works…
C0dR
  • 320
  • 1
  • 6
  • 23
0
votes
1 answer

onnx.load() | ALBert throws DecodeError: Error parsing message

Goal: re-develop this BERT Notebook to use textattack/albert-base-v2-MRPC. Kernel: conda_pytorch_p36. PyTorch 1.8.1+cpu. I convert a PyTorch / HuggingFace Transformers model to ONNX and store it. DecodeError occurs on onnx.load(). Are my ONNX files…
DanielBell99
  • 896
  • 5
  • 25
  • 57
0
votes
1 answer

Converting PyTorch to ONNX model increases file size for ALBert

Goal: Use this Notebook to perform quantisation on albert-base-v2 model. Kernel: conda_pytorch_p36. Outputs in Sections 1.2 & 2.2 show that: converting vanilla BERT from PyTorch to ONNX stays the same size, 417.6 MB. Quantization models are…
DanielBell99
  • 896
  • 5
  • 25
  • 57
0
votes
0 answers

ONNX Runtime error: node->GetOutputEdgesCount() == 0 was false. Can't remove node

I have a simple Keras RNN model, composed by embedding, LSTM, and linear layers: loaded_model.layers Out[23]: [, , …
MattS
  • 1,701
  • 1
  • 14
  • 20
0
votes
1 answer

faster_rcnn_r50 pretrained converted to ONNX hosted in Triton model server

I went through the mmdetection documentation to convert a pytorch model to onnx here link All installations are correct and i'm using onnxruntime==1.8.1, custom operators for ONNX Runtime MMCV_WITH_OPS. I'm using the…
Nrepesh Joshi
  • 27
  • 1
  • 5
0
votes
0 answers

How to force the opset version on ONNX to allow quantization?

I'm making some test converting ML models from several frameworks to ONNX, but I'm consistently getting warnings and errors related with the opset, E.G. the bellow code is for a lgmb model: import numpy as np import lightgbm as lgb import…
Luis Ramon Ramirez Rodriguez
  • 9,591
  • 27
  • 102
  • 181
0
votes
1 answer

[ONNXRuntime for android]. What am I have to do solve this problem? AssignNodesToEpsFromHashesImpl Failed to find kernel def hash

I got an error like below to use ONNXRuntime on Androidproject. [error message] AssignNodesToEpsFromHashesImpl Failed to find kernel def hash (8090321298879394920) in kernel registries for MaxPool(8) node with name 'layer3/pool_3/MaxPool'.
소문주
  • 31
  • 3
0
votes
1 answer

EntryPointNotFoundException: Unable to find an entry point named 'OrtGetApiBase' in DLL 'onnxruntime'

I am trying to run an sklearn onnx model in C# 4.8 framework on Windows 10 Pro with VS 2019, I have used the following code using examples…
azuric
  • 2,679
  • 7
  • 29
  • 44
0
votes
1 answer

How to get the language modeling loss by passing 'labels' while using ONNX inference session?

When using GPT2 we can simply pass on the 'labels' parameter to get the loss as follows: import torch from transformers import GPT2Tokenizer, GPT2LMHeadModel tokenizer = GPT2Tokenizer.from_pretrained('gpt2') model =…
Sachin Saxena
  • 11
  • 1
0
votes
1 answer

Cannot get correct predictions from ONNX model from Customvision

I am evaluating customvision.ai for training image classification model and then downloading that model as an onnx file that would be consumed within a .Net Windows forms app. I created a new project, uploaded few images, tagged them and was able to…
Nouman Qaiser
  • 283
  • 4
  • 14
0
votes
1 answer

ONNX converted TensorFlow saved model runs on CPU but not on GPU

I converted a tensorflow saved model to ONNX format using tf2onnx : python3 -m tf2onnx.convert --saved-model saved_model/ --output onnx/model.onnx --opset 11 The conversion worked fine and I can run inference with the ONNX model using CPU. I…
CDuvert
  • 367
  • 1
  • 3
  • 14
0
votes
1 answer

tf2onnx Tensorflow to Onnx inconsistent outputs

I'm experimenting with creating a super simple Tensorflow network (one data processing Lambda layer), and then converting the model to ONNX and verifying the results match when calling the ONNX model from onnxruntime. I'm using Tensorflow v2.5.0. &…
colton
  • 391
  • 1
  • 4
  • 11
0
votes
1 answer

Onnxruntime error 9: LeakyRelu not implemented

I have converted a simple pytorch model to onnx format, and failed of trying to load and evaluate the model in a separate file through onnxruntime. It gave the error message: NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find…
ywan459
  • 1
1 2 3
…
19 20