ONNX Runtime is a cross-platform inference and training machine-learning accelerator.
Questions tagged [onnxruntime]
292 questions
0
votes
1 answer
Python set ONNX runtime to return tensor instead of numpy array
In python I'm loading my predefined model (super-gradients, yolox-s):
onnx_session = onnxrt.InferenceSession("yolox_s_640_640.onnx")
Then I load some data and run it:
dataset = MyCostumeDataset(args.path, 'val')
val_dataloader = DataLoader(dataset,…
Ruslan
- 911
- 2
- 11
- 28
0
votes
0 answers
ONNX runtime predictions on raw CuPy arrays?
I am following the ONNX inference tutorial at: https://github.com/onnx/models/blob/main/vision/classification/onnxrt_inference.ipynb.
Instead of doing the pre-processing in pure NumPy, I have re-written the function to be done in CuPy for…
JOKKINATOR
- 356
- 1
- 11
0
votes
0 answers
Check and modify the weights of a layer of an onnx model at runtime
I'm using Onnx Models from ONNX Models ZOO with ONNXRuntime as a engine for inference.
I want to check and modify the weights of a convolutional layer (or any layer) of an ONNX model at runtime. This is to do it inside…
Izan C. G
- 1
- 1
0
votes
0 answers
Why is the response time of ONNX Runtime increasing?
I'm using from ONNX Runtime library to inference my deep neural network model in c++. The inference time on the CPU is about 10 milliseconds. When I use the GPU(Nvidia 1050 ti), the inference time is about 4ms for about the first minute after the…
reza
- 67
- 5
0
votes
0 answers
Extracting gradients of weight of ONNX model
I am currently working on the testing of DL libraries. Specifically, I want to extract the gradients of weights of ONNX model to do differential testing.
However, I have read a lot of posts online but still could not find a clear way to extract the…
Jerry
- 1
0
votes
0 answers
Is it possible to use model.predict on a loaded onnx model with type 'onnx.onnx_ml_pb2.ModelProto' in Python?
I have trained a model to detect faces. Now for an assignment I am trying to detect faces without a mask as an anomaly. I trained my model in the yolov5 google collab with a public dataset from Roboflow.
I tried loading the model into my python…
Luukv19
- 33
- 1
- 6
0
votes
0 answers
Optimum sentence-similarity task not supported
I'm trying to build onnx-optimized text model for calculating sentence embeddings (for further semantic similarity calculation):
import os
import torch
from transformers import AutoTokenizer
from optimum.onnxruntime import ORTOptimizer,…
Nick Zorander
- 131
- 12
0
votes
0 answers
ModuleNotFoundError: No module named 'models' on converting .pt to onnx
I am trying to convert my yolov5 model .pt file to onnx format.It shows this error
Traceback (most recent call last): File "onnx.py", line 10, in model = torch.load(weights) File…
0
votes
0 answers
run(): incompatible function arguments in onnxruntime
In doing inference on mobilenet 0.25 model-In face detection task-, I prefer to convert model to ONNX. Then, I used a simple code to do inference like this one:
import onnx
import onnxruntime
import cv2
import numpy as np
import time
import…
BarzanHayati
- 637
- 2
- 9
- 22
0
votes
1 answer
No such file or directory while compiling c++ in vscode include onnxruntime api header file
I'm very new to c++. I use VSCode and Ubuntu 18.04. I'm trying to use onnxruntime api in c++ to deploy my net model file.
Firstly I just tested including the onnxruntime api header file
#include
#include
#include
#include…
Toby_Xiang
- 1
- 1
0
votes
1 answer
how can load ONNX model in C++
I have convert PyTorch SuperResouloution (.pth) Model to ONNX model (.onnx) with code below:
import torch
import onnx
from onnx_tf.backend import prepare
from basicsr.archs.rrdbnet_arch import RRDBNet
pth_path = "RESRGAN_x4plus_Main.pth"
Model =…
0
votes
0 answers
Explanation for OpenCV resize functions' implementation
I trained my custom AI model in python, and I used OpenCV python binding to open and resizing images.
After training, I tried to do inference with test images, without using OpenCV.
That is, I tried to open and resize test images with PIL.image.
I…
happychild
- 83
- 7
0
votes
1 answer
Onnxruntime extensions are not installing on Arm Based Processor
Error when installing onnxruntime-extensions
I also tried different versions but everytime got same error
error: could not find a version that satisfies the requirement onnxruntime-extensions (from versions: none)
error: no matching distribution…
Farid
- 72
- 10
0
votes
0 answers
What is the best way to read metadata from onnx model from C++?
I have model in onnx format that contains attribute "list_classes". I run it with opencv dnn. I need to read this list using C++.
I tried opencv dnn library, but it seems like there is no instruments for that.
I tried onnxruntime, but I can't even…
0
votes
0 answers
I got always same result form RNN model exported to ONNX
I exported pytorch RNN model to ONNX with torchscript
but onnx return always same result. i got different results with pytorch, torchscript
here is my code
class RNN(nn.Module):
def __init__(self, input_size, hidden_size, num_classes,…
user1798569
- 1
- 2