Stack Overflow
Stack Overflow

Questions tagged [onnxruntime]

ONNX Runtime is a cross-platform inference and training machine-learning accelerator.

See onnxruntime github project.

292 questions
3
votes
1 answer

Upgrade ONNX model from version 9 to 11

I'm working with an ONNX model that I need to quantize in order to reduce its size, for that I'm following the instructions on the official documentation: import onnx from onnxruntime.quantization import quantize_dynamic, QuantType model_fp32 =…
Luis Ramon Ramirez Rodriguez
  • 9,591
  • 27
  • 102
  • 181
3
votes
0 answers

How to Slice ONNXRuntime Tensor?

Assume that a Microsoft.ML.Onnxruntime.Tensors.Tensor variable has been created with dimensions [d1, d2, d3]. Is there a way to return a copy or view of a slice over certain dimensions? I wanted to do the equivalent of subset =…
premes
  • 363
  • 2
  • 8
3
votes
1 answer

How install Nuget Microsoft.ML.OnnxRuntime.Gpu for ONNX Runtime in Ubuntu 18?

I'm trying to use ONNX runtime for Ubuntu: https://onnxruntime.ai/ I selected Linux, C++, x64, then CUDA. It says "Install Nuget package Microsoft.ML.OnnxRuntime.Gpu Refer to docs for requirements." When I click on "Microsoft.ML.OnnxRuntime.Gpu",…
user5739619
  • 1,748
  • 5
  • 26
  • 40
3
votes
0 answers

Scikit-Learn like models for production?

I've currently fit a model using sktime (a scikit like library) which provides interfaces to dump and load pickled models. However if I fit in x86 (64 bit OS) and run inference on raspberry pi zero which is armv6 (32 bit OS) the predictions are…
Rakshith G B
  • 816
  • 2
  • 8
  • 24
3
votes
1 answer

How can I change ONNX result = Object {Microsoft.ML.OnnxRuntime.Tensors.DenseTensor} to Something?

Hellow! I'm very new to Microsoft ML ONNX. Making GUI using 'Visual studio 2019', 'Windows Forms App' with '.NET framework(C# 7.3 ver)' I use python sklearn to make simple KNN, then export to ONNX form. MS VS Forms are receiving the Sensor data from…
JaeHan
  • 51
  • 4
3
votes
1 answer

Parse an ONNX model using C++. Extract layers, input and output shape from an onnx model using c++

I'm trying to extract data like input layers, output layers and their shapes from an onnx model. I know there is python interface to do this. I want to do something similar to this code but in c++. I have also pasted the code from the link. I have…
Hitesh Kumar
  • 193
  • 3
  • 11
3
votes
1 answer

Trying to incorporate ML onnx model to Android App

I have a trained onnx model that I want to incorporate into an android app. I'm actually working on a uni project, combining ML & Android development. After a long research, since I don't want to use a python private REST API, I came to the…
ZookKep
  • 481
  • 5
  • 13
3
votes
1 answer

How to use ONNX model in C++ code on Linux?

I train some Unet-based model in Pytorch. It take an image as an input, and return a mask. After training i save it to ONNX format, run it with onnxruntime python module and it worked like a charm. Now, i want to use this model in C++ code in…
Anton Ganichev
  • 2,184
  • 1
  • 18
  • 17
3
votes
1 answer

Error on running Super Resolution Model from ONNX

Testing ONNX model for super resolution model, I have error running this sample program. My ONNX version is 1.5.0 with onnxruntime 1.4.0. Onnxruntime was installed using pip. Pytorch version is 1.6.0 Error is at ort_session =…
batuman
  • 7,066
  • 26
  • 107
  • 229
3
votes
2 answers

Setting up ONNX Runtime on Ubuntu 20.04 (C++ API)

I am currently in the midst of trying to get my image processing programs to work on Ubuntu (coming from windows). I have successfully built and linked the OpenCV and Boost libraries to work with my cpp programs but I have yet to find any…
ApluUAlberta
  • 105
  • 1
  • 9
3
votes
1 answer

WinML inference time on GPU 3 time slower than Tensorflow python

I try to use a tensorflow model trained on python in WinML. I successfully convert protobuf to onnx. The following performance result are obtained : WinML 43s OnnxRuntime 10s Tensorflow 12s The inference on CPU take arround 86s. On performance…
Erwan
  • 587
  • 4
  • 23
3
votes
3 answers

Quantization of Onnx model

I am trying to quantize an ONNX model using the onnxruntime quantization tool. My code is below for quantization: import onnx from quantize import quantize, QuantizationMode # Load the onnx model model =…
Parag Jain
  • 612
  • 2
  • 14
  • 31
2
votes
0 answers

How to use TopK operator in ONNX?

I am writing a converter and calculator to convert my custom sklearn transformers into ONNX models. I need to calculate the median of my data points. Interesting point - ONNX has no function to calculate the median (at least I didn't find anything…
paradocslover
  • 2,932
  • 3
  • 18
  • 44
2
votes
1 answer

Failed to use OnnxRuntime on Windows 7

Here's an issue we are encountering when using OnnxRuntime on Windows 7. We have provided below the software environment of our machine for your reference. Software Environment Windows 10 Visual Studio 2022 dotnet 6.0 Microsoft.ML.OnnxRuntime…
Shinohara Haruna
  • 31
  • 5
2
votes
0 answers

How can I execute decoder of ONNX Export from Seq2Seq model

I made an export of the Helsinki model using python optimum and i am trying to run the model with only the onnx environment and implement beam search from scratch because I have to later port this to a system not running python. So I want to…
klsmgföl
  • 21
  • 3
1
2
3
…
19 20