1

I'm running the TfLite model analyzer on Google Brain's AutoML EfficientDet model like so:

model_path = "/content/efficientdet.tflite"
tf.lite.experimental.Analyzer.analyze(model_path=model_path, gpu_compatibility=True)

The analyzer output shows that the last layer is not a supported TfLite GPU operation:

GPU COMPATIBILITY WARNING: Not supported custom op TFLite_Detection_PostProcess
GPU COMPATIBILITY WARNING: Subgraph#0 has GPU delegate compatibility issues at nodes 482 with TFLite runtime version 2.8.2

The TFLite_Detection_PostProcess operator is not supported on the GPU. This is causing CPU/GPU synchronization issues and it is increasing the latency.

What do I need to do to get the op running on the GPU in Tensorflow Lite?

I'm open to re-compiling the TfLite binary or modifying the neural network.

Fred
  • 381
  • 5
  • 17

1 Answers1

-1

Suggested workaround will be to use 2.11 TFLite Runtime and efficientdet model from Tensorflow hub which are regularly tested and maintained.

Attached gist for reference.

Manu Kowsik 1
  • 1
  • I don't think your solution works. It's showing this in the gist: `GPU COMPATIBILITY WARNING: Not supported custom op TFLite_Detection_PostProcess` – Fred Feb 20 '23 at 22:30
  • 1
    Yeah, saw that for node 266. I think we can disable quantization for the layers having gpu compatibility through [tfl.quantization_debugger](https://www.tensorflow.org/lite/performance/quantization_debugger)/[tfmot](https://www.tensorflow.org/model_optimization/api_docs/python/tfmot) api . will come with a updated gist later. Thank you! – Manu Kowsik 1 Feb 21 '23 at 05:06
  • 1
    Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Feb 24 '23 at 12:33