0

We are interested in using TFlite with Python and with GPU support. How can we configure TFlite in Python to enable the GPU delegate? If it cannot be done currently, what should we change in TFLite to allow Python to use the GPU delegate?

It is worth mentioning that we are able to successfully use a GPU with TFlite and C++.
Following the instructions here, we built TFlite with GPU support.

We found that to use the GPU with TFlite in C++, you first need to configure the GPU delegate, as explained here.

It looks something like this:
TfLiteGpuDelegateOptionsV2 options = TfLiteGpuDelegateOptionsV2Default(); auto* delegate = TfLiteGpuDelegateV2Create(const_cast<TfLiteGpuDelegateOptionsV2 *>(&options)); if (interpreter->ModifyGraphWithDelegate(delegate) != kTfLiteOk) return false;

We are able to build the TFlite Python runtime with GPU support enabled, but we do not see any Python APIs that allow us to enable the GPU delegate as in C++.

Robert Foley
  • 31
  • 4
  • 1
    Hi @Robert! You can load the bazel built GPU delegates using [load_delegate](https://www.tensorflow.org/api_docs/python/tf/lite/experimental/load_delegate) api and use [analyzer](https://www.tensorflow.org/api_docs/python/tf/lite/experimental/Analyzer) to see whether model is compatible with GPU or not . –  Jan 03 '23 at 18:08
  • Also searched for quite a while and could not find Python code that uses TfLite + GPU. – Bart Jul 01 '23 at 10:39

0 Answers0