Stack Overflow
Stack Overflow

Questions tagged [onnxruntime]

ONNX Runtime is a cross-platform inference and training machine-learning accelerator.

See onnxruntime github project.

292 questions
2
votes
1 answer

Export MarianMT model to ONNX

I would like to use the Helsinki-NLP/opus-mt-de-en model from HuggingFace to translate text. This works fine with the HuggingFace Inference API or a Transformers pipeline, e.g.: from transformers import AutoTokenizer, pipeline from…
RGe
  • 1,181
  • 1
  • 10
  • 19
2
votes
0 answers

ONNX sklearn model interface for android only returns class label as output

I am trying to get probability/confidence score from the model on my android application. While creating the ONNX model in python, I am able to see the confidence/probability score for a given input. However, when I give the same input on Android,…
ImMathan
  • 3,911
  • 4
  • 29
  • 45
2
votes
1 answer

Why Onnxruntime runs 2-3x slower in C++ than Python?

I have a code that runs 3 inference sessions one after the other. The problem that I am having is that it only runs at top performance in my Mac and the Windows VM (VMWare) that runs in my Mac. It takes between 58-68s to run my test set. When I ask…
Adriel Jr
  • 2,451
  • 19
  • 25
2
votes
0 answers

How to post-process a Yolov7 ONNX model?

I trained a YOLOv7 model on a custom dataset and converted it to ONNX. The input of the model on Netron reads "Float32(1,3,640,640)" which I understand. The output, however, is unclear to me as other tutorials mentioned there should be 6 elements…
Loay Altal
  • 21
  • 3
2
votes
0 answers

ONNX Runtime: If no GPU, gracefully use CPU instead?

I've only used ORT for CPU. It looks like there is a separate NuGet package for the ORT for GPU. If I use the GPU version, if there is no GPU on the machine its running on, will it "Gracefully" run on CPU instead? I would think almost everyone…
Tullhead
  • 565
  • 2
  • 7
  • 17
2
votes
2 answers

ONNX Runtime JS how to use Tensors as multi-dim arrays?

This also would seem something trivial but I've been searching the web and experimenting for a frustrating amount of time. In onnx runtime web I create a tensor, i.e: const a = new ort.Tensor('float32', array, [2, 10]) With array being some…
Juan Alberto López Cavallotti
  • 4,644
  • 3
  • 25
  • 43
2
votes
0 answers

TypeError: can‘t pickle onnxruntime.capi.onnxruntime_pybind11_state.InferenceSession objects

I Using flask to provide model services, this problem is encountered when using multiple processes to speed up obtaining model results, then the model initialization method is also passed to the pool, but it takes time. How to initialize the model…
pluto
  • 21
  • 1
2
votes
0 answers

How to quantize an ONNX model converted from a XGBoost classifier model?

I converted a XGBoost classifier model to an ONNX model by onnxmltools and quantized the ONNX model using ONNX quantize_dynamic(). But I didn't get a quantized ONNX model with smaller model file size or faster inference time. I used Anaconda3,…
SC Chen
  • 23
  • 5
2
votes
1 answer

onnxruntime: Given model could not be parsed while creating inference session. Error message: Protobuf parsing failed

According to the example code mentioned below the library. I have followed the example code but it didn't work. [Library] https://github.com/notAI-tech/NudeNet/ Code from nudenet import NudeClassifier import onnxruntime classifier =…
Khawar Islam
  • 2,556
  • 2
  • 34
  • 56
2
votes
0 answers

Converting Pth to onnx - convolution_mode error

I am trying to convert .pth to .onnx file, class Net(nn.Module): def __init__(self): super(Net, self).__init__() cnn = nn.Sequential() cnn.add_module('c1', nn.Conv2d(3, 32, 3, 1, 1)) cnn.add_module('r1',…
Vishak Raj
  • 141
  • 1
  • 1
  • 8
2
votes
1 answer

Couldn't convert pytorch model to ONNX

I used this repo : https://github.com/Turoad/lanedet to convert a pytorch model that use mobilenetv2 as backbone To ONNX but I didn't succeeded. i got a Runtime error that says: RuntimeError: Exporting the operator eye to ONNX opset version 12…
Abdul Hakawati
  • 23
  • 4
2
votes
1 answer

RuntimeError: Only tuples, lists and Variables are supported as JIT inputs/outputs. Dictionaries and strings are also accepted

I was trying to convert my pytorch model to onnx but I am facing RuntimeError: Only tuples, lists and Variables are supported as JIT inputs/outputs. Dictionaries and strings are also accepted, but their usage is not recommended. Here, received an…
user16668992
2
votes
0 answers

NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from onnx/bert-base-cased/model.onnx failed:Load model onnx/bert-base-cased/model.onnx

Goal: to successfully save and load in a HuggingFace NLP model. Kernel: conda_pytorch_p36. I performed Restart & Run All, and refreshed file view in working directory. I'm following along with this code tutorial, the first Python code…
DanielBell99
  • 896
  • 5
  • 25
  • 57
2
votes
2 answers

Is it possible to train ONNX models developed in tensorflow and pytorch with C++?

I wonder if its possible to use tensorflow and pytorch models converted to onnx models to train them with the C++ Api like it is done in e.g. https://gist.github.com/asimshankar/5c96acd1280507940bad9083370fe8dc with a tensorflow model. I just found…
Usaint
  • 33
  • 6
2
votes
0 answers

Is there a way to include a model description when using `torch.onnx.export`?

Problem description We would like our onnx models to have some form of description, and ideally some other metadata including our internal version number. Currently we train with pytorch lightning and use onnxruntime for inference. Below is a…
this_josh
  • 333
  • 2
  • 11
1 2
3
…
19 20