Stack Overflow
Stack Overflow

Questions tagged [onnxruntime]

ONNX Runtime is a cross-platform inference and training machine-learning accelerator.

See onnxruntime github project.

292 questions
2
votes
1 answer

ONNX Runtime C# does not remember the state of LSTM networks

I exported a trained LSTM neural network from this example from Matlab to ONNX. Then I try to run this network with ONNX Runtime C#. However, it looks like I am doing something wrong and the network does not remember its state on the previous…
Pavel Gorev
  • 63
  • 7
2
votes
1 answer

Error Making prediction with python onnxruntime

I have created an very basic decision tree using the sklearn library. This tree is trained based on 4 features: feat1 INT feat2 INT feat3 FLOAT feat4 FLOAT And the label/target feature is a boolean value (0 or 1). I converted the tree into a ONNX…
user7432713
  • 197
  • 3
  • 17
1
vote
1 answer

Load ONNX model with low-bit quantized weight

I have quantized my model to 2-bit and packed them into uint8 format (store 4x 2-bit weight in an uint8 variable) in pytorch. In this way, the model size has been reduced from 1545M to 150M, and the VRAM for loading the model is also greatly reduced…
Yefei He
  • 11
  • 1
1
vote
2 answers

Using a pre-trained exported Pytorch resnet18 model with ONNX

I'm fairly new to deep learning and I've managed to train a resnet18 model with FastAI for multilabel prediction. learn = cnn_learner(dls, resnet18, metrics=partial(accuracy_multi, thresh=0.2)) Next, I exported the model to…
Denis Nutiu
  • 1,178
  • 15
  • 22
1
vote
0 answers

Can't install onnxruntime

I want to install the onnxruntime pip library but i have this output: pip install onnxruntime ERROR: Could not find a version that satisfies the requirement onnxruntime (from versions: none) ERROR: No matching distribution found for…
Sebastian Viollaz
  • 11
  • 2
1
vote
1 answer

Load onnx model in browser, can't find wasm file

Just trying to load an .onnx model that I have into browser using onnxruntime-web The code is run in a React app using a vite server, import * as ort from 'onnxruntime-web'; const App = () => { const create = async () => { const…
jojeyh
  • 276
  • 1
  • 3
  • 12
1
vote
1 answer

Run Rust binary without `cargo run`

I can compile and run my Rust project without any problems using cargo run --release. As a second step I simply want to create the binary using cargo build --release and then executing it by running ./target/release/crate_name (it is also explained…
nerdizzle
  • 424
  • 4
  • 17
1
vote
0 answers

ONNXRUNTIME-GPU with multiprocessing and multigpu

Suppose that I have 4 Process and 2 GPUs. Running the following code, it works while n = 1. If I set n = 2, it would be blocked at the second loop. import onnxruntime as ort from multiprocessing import Process, Queue class Onnx: def…
walstruzz
  • 11
  • 1
1
vote
0 answers

Change input and output shapes in onnx model

I'm quite new to machine learning, so I apologize in advance for the question. I have a pre-trained onnx model, with defined input and output shapes. Is it possible to change those values? I looked at possible solutions, trying to use for example…
Montparg
  • 63
  • 7
1
vote
0 answers

TensorRT Inference is giving partial output

Description (Already posted in Nvidia forum, but posting here for quicker responses. Ref: https://github.com/NVIDIA/TensorRT/issues/2795) I am using a ONNX model which take input (n,2) as input and returns the negation of the input as output. Ex:…
user2773716
  • 865
  • 1
  • 7
  • 5
1
vote
2 answers

How to do inference with YOLOv5 and ONNX

I've trained a YOLOv5 model and it works well on new images with yolo detect.py I've exported the model to ONNX and now i'm trying to load the ONNX model and do inference on a new image. My code works but I don't get the correct bounding boxes. I…
Timothee W
  • 149
  • 7
1
vote
1 answer

Trying to use onnxruntime with GPU. SessionOptionsAppendExecutionProvider_CUDA gives error

I'm learning onnxruntime inferencing with GPU. I got the code from https://github.com/microsoft/onnxruntime-inference-examples.git. I tried the case main/c_cxx/fns_candy_style_transfer. Follow instructions in…
Yanhua Cui
  • 11
  • 2
1
vote
1 answer

Why does YoloV8 perform poorly when exported to .onnx and run with onnxruntime or opencv dnn? The results just don't compare to torch .pt model files

I'm working on transfer learning a coco trained yolov8 model to detect objects in an entirely different use case. I get really encouraging performance metrics when I reload the trained model from its model.pt file using the ultralytics library and…
moonboi
  • 21
  • 2
1
vote
1 answer

Pytorch runtime not included when project is built as a jar

We have our ONNX model, and we're trying to use it in our app. Running the project from the IDE works just fine, but running the jar file produces this error. We debugged this issue and found that the supportedEngines in IDE runtime includes…
HishamGarout
  • 1,743
  • 3
  • 11
  • 12
1
vote
0 answers

In pytorch torchscript, how to define mutiple entry point

Have a torch model as follow : MyModel update(self) : Update some params. predict(self,X) : Predict with some input tensor. When exporting to torchscript, is there a way to have 2 entry points: One for update() One for…
tensor
  • 3,088
  • 8
  • 37
  • 71
1 2 3
…
19 20