Stack Overflow
Stack Overflow

Questions tagged [nnapi]

The Android Neural Networks API (NNAPI) is an Android C API designed for running computationally intensive operations for machine learning on mobile devices.

28 questions
2
votes
1 answer

how to use tensorflow-lite nnapi on android 7 & 8

NNAPI is available on android 8.1. But I want to use the NNAPI on android 7&8(arm64). the NNAPI is used by tensorflow-lite. Where can I download libneuralnetworks.so?
xhsoldier
  • 575
  • 6
  • 31
1
vote
1 answer

"Tensorflow + NNAPI" runs very slow on Samsung Galaxy S21

Samsung Galaxy S21 is the only device runs slow when using tensor-flow lite and NNAPI together. Did anyone encounter this issue before? Background I used tensor-flow lite and NNAPI and built my app on different Android devices. Samsung Galaxy S21 is…
Di Wang
  • 471
  • 1
  • 8
  • 22
1
vote
1 answer

Tensorflow Lite Android: Both GPU delegate and NNAPI delegate are slower than CPU

I am currently evaluating and comparing the performance of some tensorflow models on different smartphones. I am testing the MNIST and CIFAR10 databases. The strange thing is, when i try to speed up the inference times with hardware acceleration,…
stefferino
  • 11
  • 4
1
vote
0 answers

Does Hiai foundation available for native shell program?

I am interested alot in huawei NPU(I am using kirin9000, 990, 810) and I just found that when using hiai foundation api to use NPU, hiai foundation is not working when i build as native shell program through adb shell, however, it is working well…
JanosLim
  • 11
  • 3
1
vote
1 answer

How can I let Kirin 990's NPU work on TensorFlow Lite?

I successfully converted TensorFlow model to TensorFlow Lite float16 model according to Post-training float16 quantization. The below is a diagram of the converted model. And I ran successfully it on MatePad Pro(Kirin 990) by my C++ code. What I…
Yuji
  • 614
  • 4
  • 18
1
vote
0 answers

NHWC vs NCHW benefits

I believe I understand the differences between NHWC and HCHW. NHWC (batch, height, width, channels) has output index order of, assuming 3 input channels representing RGB, R0, G0, B0, R1, G1, B1, ..., Rc, Gc, Bc. NCHW (batch, channels, height,…
grizzleKat45
  • 23
  • 4
1
vote
1 answer

Use ML Kit with NNAPI

I'm trying to use the new Google machine learning sdk, ML Kit, on an Android devices that run Android 9. From the official site: ML Kit makes it easy to apply ML techniques in your apps by bringing Google's ML technologies, such as the Google…
pado
  • 123
  • 1
  • 11
1
vote
1 answer

tensorflow-lite and nnapi hw acceleration for embedded linux

TensorFlow-lite is for embedded linux system and Android NNAPI is for hw acceleration for Android system. I do like to find out solutions for embedded linux not android with Tensorflow-Lite hw acceleration.
K Lee
  • 21
  • 2
0
votes
0 answers

Using NnApiDelegate in TFLite 2.11.0 returns same embeddings for all images. It works fine for 2.6.0

In my android app I'm using Facenet model to recognize faces. I have added NnApiDelegate to the interpreterOptions. My app has been working with TFLite version 2.6.0. When I upgraded the TFLite version to 2.10.0 or 2.11.0, I see that the model…
Shubham Agarwal
  • 95
  • 6
0
votes
0 answers

tflite + NNAPI stops working when starting processes inside threads

I am working on a script that runs on a iMX8MP board equipped with NPU. The script acquires and processes images on the NPU using tflite_runtime with NNAPI delegate. Sometimes the script starts threads to perform other parallel tasks. I noticed that…
firion
  • 296
  • 3
  • 12
0
votes
0 answers

How to implement monitoring of GPU and NPU usage / performance in Android?

I'm using the NNAPI delegate of Tensorflow Lite to run our net. I am trying to determine which accelerator (NPU, GPU, DSP) is used during runtime. In the Burnout Benchmark there are three graphs that show the usage of CPU, GPU and NPU. I know how to…
iscod
  • 21
  • 2
0
votes
1 answer

Execute different operations of a NN model graph with different hardware accelerators while using Onnxruntime

I was trying to utilize NNAPI via OnnxRuntime for NN model inferencing on an Android device. Based on the youtube video here: https://www.youtube.com/watch?v=Ij5MoUnLQ0E it is possible to specify the hardware accelerators for operators in the model.…
ashwinjoseph
  • 359
  • 3
  • 12
0
votes
1 answer

Access Android C++ NNAPIs - ANeuralNetworksCompilation_createForDevices gives error

Trying to Access GPU using Android C++ NNAPIs ANeuralNetworksCompilation_createForDevices gives error ANeuralNetworksCompilation_createForDevices gives error Linking fails.
Mohammed Siddiqh
  • 1
  • 2
0
votes
1 answer

tflite model outputs different predictions on CPU and NNAPI

I trained and quantized a Tensorflow model on a Ubuntu 18.04 machine and I converted it to tflite format. Then I deployed it on a Linux Yocto board equipped with a NPU accelerator, tflite_runtime and NNAPI. I noticed that the same tflite model…
firion
  • 296
  • 3
  • 12
0
votes
1 answer

How do I use HAL version 1.2 on Tensorflow Lite for Android?

I have a quantized TensorflowLite model that I'm loading onto a Pixel 3 running Android 11. I built the model using Tensorflow Lite 2.5 and I'm using the nightly builds of Tensorflow for Android. I'm initializing the TFLite Interpreter using the…
Jeff
  • 5,746
  • 4
  • 33
  • 40
1
2