Onnxruntime error 9: LeakyRelu not implemented

Question

I have converted a simple pytorch model to onnx format, and failed of trying to load and evaluate the model in a separate file through onnxruntime. It gave the error message:

NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find an implementation for the node LeakyRelu_1:LeakyRelu(6)

But the docs clearly lists the leakyrelu operator: https://github.com/microsoft/onnxruntime/blob/master/docs/OperatorKernels.md

The script for converting is super simple.

import torch.onnx
import torch

torch.set_default_dtype(torch.float64)

Model = torch.load("Test.pth")
inputs = torch.randn(1,2)
torch.onnx.export(Model,inputs, "Test.onnx", export_params=True)

the script for loading the converted model is also quite simple,

import onnx
import onnxruntime as rt
import numpy as np


onnx_model = onnx.load("Test.onnx")
onnx.checker.check_model(onnx_model)

#print(onnx_model.graph)
print("[Graph Input] name: {}, shape: {}".format(onnx_model.graph.input[0].name,
                                                 [dim.dim_value for dim in onnx_model.graph.input[0].type.tensor_type.shape.dim]))
print("[Graph Output] name: {}, shape: {}".format(onnx_model.graph.output[0].name,
                                                  [dim.dim_value for dim in onnx_model.graph.output[0].type.tensor_type.shape.dim]))


print(onnx.helper.printable_graph(onnx_model.graph))

sess = rt.InferenceSession("Test.onnx")

The printed graph is

graph torch-jit-export (
  %input.1[DOUBLE, 1x2]
) initializers (
  %0.weight[DOUBLE, 228x2]
  %0.bias[DOUBLE, 228]
  %2.weight[DOUBLE, 70x228]
  %2.bias[DOUBLE, 70]
  %4.weight[DOUBLE, 1x70]
  %4.bias[DOUBLE, 1]
) {
  %7 = Gemm[alpha = 1, beta = 1, transB = 1](%input.1, %0.weight, %0.bias)
  %8 = LeakyRelu[alpha = 0.00999999977648258](%7)
  %9 = Gemm[alpha = 1, beta = 1, transB = 1](%8, %2.weight, %2.bias)
  %10 = LeakyRelu[alpha = 0.00999999977648258](%9)
  %11 = Gemm[alpha = 1, beta = 1, transB = 1](%10, %4.weight, %4.bias)
  return %11
}

I am wondering whether the leakyrelu has been implemented, or I have just missed something in the conversion. Thanks for the helps!

score 0 · Answer 1 · answered Jun 28 '21 at 19:13

According to the docs https://github.com/microsoft/onnxruntime/blob/master/docs/OperatorKernels.md, LeakyRelu is only implemented for type float (32-bit), while you have double (64 bit).

You may try to convert to 32-bit float right before LeakyRely in your PyTorch code. And maybe create an issue on the ONNX Runtime Github to add double support for LeakyRelu.

Onnxruntime error 9: LeakyRelu not implemented

1 Answers1