Hi I am using TensorRT for an image detection in python but getting this issue. we have tested this on Linux and working well but got issues on windows. This is issue of only getting on python , C++ inference working smoothly. I am also attaching yoloV5n6.trt.onnx file for further tests. Please help me out in this Error.
Yolort ONNX file
I am using Yolort to infer image. https://github.com/zhiqwang/yolov5-rt-stack
import torch
from yolort.runtime import PredictorTRT
# Load the serialized TensorRT engine
engine_path = "yolov5n6.engine"
device = torch.device("cuda")
y_runtime = PredictorTRT(engine_path, device=device)
# Perform inference on an image file
predictions = y_runtime.predict("new_york.jpg")
print(predictions)
Here is my environment
>python -m torch.utils.collect_env
Collecting environment information...
PyTorch version: 1.11.0+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A
OS: Microsoft Windows 10 Home
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.23.0
Libc version: N/A
Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19041-SP0
Is CUDA available: True
CUDA runtime version: 11.6.124
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU
Nvidia driver version: 511.65
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\cudnn_ops_train64_8.dll
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.11.0+cu113
[pip3] torchaudio==0.11.0+cu113
[pip3] torchvision==0.12.0+cu113
[conda] blas 1.0 mkl
[conda] cudatoolkit 11.3.1 h59b6b97_2
[conda] libblas 3.9.0 12_win64_mkl conda-forge
[conda] libcblas 3.9.0 12_win64_mkl conda-forge
[conda] liblapack 3.9.0 12_win64_mkl conda-forge
[conda] mkl 2021.4.0 h0e2418a_729 conda-forge
[conda] mkl-service 2.4.0 py39h6b0492b_0 conda-forge
[conda] mkl_fft 1.3.1 py39h0cb33c3_1 conda-forge
[conda] mkl_random 1.2.2 py39h2e25243_0 conda-forge
[conda] mypy_extensions 0.4.3 py39hcbf5309_5 conda-forge
[conda] numpy 1.22.3 pypi_0 pypi
[conda] numpy-base 1.20.3 py39hc2deb75_0
[conda] numpydoc 1.2.1 pyhd8ed1ab_2 conda-forge
[conda] pytorch 1.11.0 py3.9_cuda11.3_cudnn8_0 pytorch
[conda] pytorch-mutex 1.0 cuda pytorch
[conda] torchaudio 0.11.0 py39_cu113 pytorch
[conda] torchvision 0.12.0 py39_cu113 pytorch
Error
Loading yolov5n6.engine for TensorRT inference...
[04/24/2022-19:20:14] [TRT] [I] [MemUsageChange] Init CUDA: CPU +481, GPU +0, now: CPU 7553, GPU 1144 (MiB)
[04/24/2022-19:20:14] [TRT] [I] Loaded engine size: 12 MiB
[04/24/2022-19:20:14] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.8.0 but loaded cuBLAS/cuBLAS LT 11.5.1
[04/24/2022-19:20:14] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +821, GPU +316, now: CPU 8433, GPU 1473 (MiB)
[04/24/2022-19:20:15] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +568, GPU +264, now: CPU 9001, GPU 1737 (MiB)
[04/24/2022-19:20:15] [TRT] [W] TensorRT was linked against cuDNN 8.3.2 but loaded cuDNN 8.2.0
[04/24/2022-19:20:15] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +12, now: CPU 0, GPU 12 (MiB)
[04/24/2022-19:20:15] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.8.0 but loaded cuBLAS/cuBLAS LT 11.5.1
[04/24/2022-19:20:15] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +10, now: CPU 8988, GPU 1751 (MiB)
[04/24/2022-19:20:15] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +2, GPU +8, now: CPU 8991, GPU 1759 (MiB)
[04/24/2022-19:20:15] [TRT] [W] TensorRT was linked against cuDNN 8.3.2 but loaded cuDNN 8.2.0
[04/24/2022-19:20:15] [TRT] [E] 1: [graphContext.h::nvinfer1::rt::MyelinGraphContext::MyelinGraphContext::24] Error Code 1: Myelin (Compiled against cuBLASLt 10.2.2.0
but running against cuBLASLt 11.5.1.0.)
