Stack Overflow
Stack Overflow

Questions tagged [onnxruntime]

ONNX Runtime is a cross-platform inference and training machine-learning accelerator.

See onnxruntime github project.

292 questions
1
vote
1 answer

ONNX runtime bert inference: RuntimeError: Input must be a list of dictionaries or a single numpy array for input 'attention_mask'

I am trying to use Huggingface Bert model using onnx runtime. I have used the the docs to convert the model and I am trying to run inference. My inference code is: from transformers import BertTokenizer, BertModel, BertTokenizerFast import…
SajanGohil
  • 960
  • 13
  • 26
1
vote
0 answers

Onnx data transformations (eg. drop column)

I have a sklearn python model that takes in a pandas dataframe. I do some data transformations like dropping columns and filling missing values before inference. I have to serialize my model and do predictions in a .net application. I havent found a…
gap210
  • 55
  • 7
1
vote
0 answers

Differentiate ONNX objects

I would like to make an onnx model differentiable. As I understand exporting to ONNX does not export the AutoGrad graph, is there anyway to reconstruct if after loading? I am aware of torch-ort but to me it looks like it only works with nn.Module…
ipcamit
  • 330
  • 3
  • 16
1
vote
1 answer

How to run u2net model with ONNX in browser (client side)?

I am trying to run u2net model in browser, I have converted the pytorch u2netp model into ONNX model and wrote the following code to run it but the results very poor. I followed the same preprocessing steps as that of python script but did not get…
Swapnil Gautam
  • 21
  • 3
1
vote
0 answers

OnnxRuntime using and sklearn SVMRegressor in .NET

I have created an svmregressor using sklearn and skl2onnx: initial_type = [('double_input', DoubleTensorType([None, len(d)]))] onx = convert_sklearn(results[i].estimator_, initial_types=initial_type) with open(r"C:\my folder\x_" + str(i) +…
azuric
  • 2,679
  • 7
  • 29
  • 44
1
vote
1 answer

final_types for RandomForestClassifier skl2onnx

How is the best way to define final_types for RandomForestClassifier? If I do the following: initial_type = [('input', FloatTensorType([None, 13]))] final_type = [('output', FloatTensorType([None, 1]))] sklonnx = convert_sklearn(rfc,…
Anjoys
  • 89
  • 10
1
vote
1 answer

Memory leak (CPU's RAM) when using onnxruntime on GPU

I'm using the Insightface library from Pypi (https://pypi.org/project/insightface/), the source code is here: https://github.com/deepinsight/insightface/blob/master/python-package/insightface/model_zoo/scrfd.py. When I run it on my GPU there is a…
Yinon_90
  • 1,315
  • 12
  • 17
1
vote
1 answer

Inferencing tensorflow-trained model using ONNX in C++?

I am trying to recreate the work done in this video, CppDay20Interoperable AI: ONNX & ONNXRuntime in C++ (M. Arena, M.Verasani).The github repository for the demo code is here .So far I have trained a regression model using TensorFlow and have…
JINU RAJ
  • 63
  • 1
  • 7
1
vote
1 answer

Couldn't export Pytorch model to ONNX

I’m trying to export my trained model that in .pt format to ONNX but after I execute the script the program output bunch of logging stuff like in the pic below and the converted onnx format seems not appear in my local disk I don’t know what I got…
frankenstein
  • 125
  • 2
  • 11
1
vote
1 answer

Adding onnxruntime Library to a .Net Core Docker Container for Linux

I am trying to deploy a game server on a .NET 5.0 Docker container for Linux deployment. My current Dockerfile looks like this, and it has been working fine so far. My code is packaged into a dll that is in the Darkrift_Server_Local/Plugins folder.…
Prateek Shah
  • 23
  • 4
1
vote
1 answer

Failure ONNX InferenceSession ONNX model exported from PyTorch

I am trying to export a custom PyTorch model to ONNX to perform inference but without success... The tricky thing here is that I'm trying to use the script-based exporter as shown in the example here in order to call a function from my model. I can…
Jules
  • 395
  • 4
  • 17
1
vote
1 answer

onnxruntime with use_dml mix build hanppen error Having memory pattern enabled is not supported while using the DML Execution Provider

in window environment,is build onnxruntime with --use_dml,my command is " .\build.bat --update --build --build_shared_lib --build_wheel --config RelWithDebInfo --use_dml --cmake_generator "Visual Studio 16 2019" --parallel --skip_tests". in the…
wwbnjs
  • 21
  • 5
1
vote
1 answer

ONNX Quantized Model Type Error: Type 'tensor(float16)'

I converted onnx model from float32 to float16 by using this script. from onnxruntime_tools import optimizer optimized_model = optimizer.optimize_model("model_fixed.onnx", model_type='bert_tf', num_heads=12, hidden_size=768,…
Pranali Kulkarni
  • 21
  • 4
1
vote
0 answers

Running threads in parallel takes more time then sequential execution in python

I have two ONNX deep learned models. I want to run both the models parallelly. I am using threads from python. But surprisingly it is taking more time then running both the models sequentially. Task to be done. make a class of model load both the…
Abhishek Gangwar
  • 1,697
  • 3
  • 17
  • 29
1
vote
0 answers

How do I convert to ONNX a Spark ML model with multiple input columns and use it for scoring dynamic batch size?

I converted a logistic regression model with dynamic batch size from Spark ML to ONNX using this: initial_types = [('Features', FloatTensorType([None, 5]))] onnx_model = convert_sparkml(s_clf, 'Occupancy detection Pyspark Logistic Regression model',…
Adi E
  • 478
  • 2
  • 4
  • 9
1 2 3
…
19 20