1

I converted a TensorFlow Model to ONNX using this command: python -m tf2onnx.convert --saved-model tensorflow-model-path --opset 10 --output model.onnx The conversion was successful and I can inference on the CPU after installing onnxruntime.

But when I create a new environment, install onnxruntime-gpu on it and inference using GPU, I get different error messages based on the model. E.g. for MobileNet I receive W:onnxruntime:Default, cuda_execution_provider.cc:1498 GetCapability] CUDA kernel not supported. Fallback to CPU execution provider for Op type: Conv node name: StatefulPartitionedCall/mobilenetv2_1.00_224/Conv1/Conv2D

I tried out different opsets. Does someone know why I am getting errors when running on GPU

oezguensi
  • 930
  • 1
  • 12
  • 23

1 Answers1

1

That is not an error. That is a warning and it is basically telling you that that particular Conv node will run on CPU (instead of GPU). It is most likely because the GPU backend does not yet support asymmetric paddings and there is a PR in progress to mitigate this issue - https://github.com/microsoft/onnxruntime/pull/4627. Once this PR is merged, these warnings should go away and such Conv nodes will run on the GPU backend.

Hariharan Seshadri
  • 106
  • 2
  • 1
    Also - for future reference, might be slightly more quicker to just post an issue on the onnxruntime github repo to get a response. Thanks! – Hariharan Seshadri Aug 11 '20 at 00:12
  • Thanks for the answer. Does that mean all of the model runs on CPU or only that specific layer? – oezguensi Aug 11 '20 at 10:03
  • 1
    Sure. It only means those specific Conv nodes with assymmetric padding drop down to CPU. In your case, this is the specific node: 'StatefulPartitionedCall/mobilenetv2_1.00_224/Conv1/Conv2D'. You will find one such message for every node that has dopped down. Also, after the PR gets merged this should go away. – Hariharan Seshadri Aug 11 '20 at 23:44