TensorFlow Lite model not working in Android (Java) but works on Python

Question

Background

I successfully converted my YOLOV5 model from YOLOV5.pt to YOLOV5-fp16.tflite and was able to run it on python(refer to the test video on Google Drive). I used the conversion method python export.py --include tflite.

Google Colab where the weights.pt was trained on

I then used this and tried to port it to Android Studio(Artic Fox), loading the model to assets, adjusted the modelPath under PreviewImageActivity.java. After this, the application is then exported into an APK.

Problem

When the application is executed it cannot get past the part after the video is processed into images for object detection and crashes.

I tried searching online for solutions and it did not help as my situation seems to be different. Any help would be good since I am doing this for my school project thanks!

Here is the zip to the application with the test video to process. The Username and Password is admin and password respectively.

Edit 1

Here is the log cat error when I run the program on my device(Samsung Tab A) via USB debugging.

2022-01-19 10:06:53.117 6044-6044/bline.detector.com.ffmpegvideoeditor D/PreviewImageAdapter: bitmap: [android.graphics.Bitmap@65ea227, android.graphics.Bitmap@8e60e7d, android.graphics.Bitmap@40939c3]
2022-01-19 10:06:53.148 6044-6044/bline.detector.com.ffmpegvideoeditor D/PreviewImageAdapter: bitmap: [android.graphics.Bitmap@65ea227, android.graphics.Bitmap@8e60e7d, android.graphics.Bitmap@40939c3, android.graphics.Bitmap@bca1579]
2022-01-19 10:06:53.180 6044-6044/bline.detector.com.ffmpegvideoeditor D/PreviewImageAdapter: bitmap: [android.graphics.Bitmap@65ea227, android.graphics.Bitmap@8e60e7d, android.graphics.Bitmap@40939c3, android.graphics.Bitmap@bca1579, android.graphics.Bitmap@33baf1f]
2022-01-19 10:06:53.210 6044-6044/bline.detector.com.ffmpegvideoeditor D/PreviewImageAdapter: bitmap: [android.graphics.Bitmap@65ea227, android.graphics.Bitmap@8e60e7d, android.graphics.Bitmap@40939c3, android.graphics.Bitmap@bca1579, android.graphics.Bitmap@33baf1f, android.graphics.Bitmap@e4d9035]
2022-01-19 10:06:53.423 6044-6044/bline.detector.com.ffmpegvideoeditor D/AndroidRuntime: Shutting down VM
2022-01-19 10:06:53.427 6044-6044/bline.detector.com.ffmpegvideoeditor E/AndroidRuntime: FATAL EXCEPTION: main
    Process: bline.detector.com.ffmpegvideoeditor, PID: 6044
    java.lang.RuntimeException: Unable to start activity ComponentInfo{bline.detector.com.ffmpegvideoeditor/bline.detector.com.ffmpegvideoeditor.activity.PreviewImageActivity}: java.lang.IllegalArgumentException: Cannot copy from a TensorFlowLite tensor (StatefulPartitionedCall:0) with shape [1, 6300, 6] to a Java object with shape [1, 10, 4].
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3623)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3775)
        at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:83)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2261)
        at android.os.Handler.dispatchMessage(Handler.java:107)
        at android.os.Looper.loop(Looper.java:237)
        at android.app.ActivityThread.main(ActivityThread.java:8107)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
     Caused by: java.lang.IllegalArgumentException: Cannot copy from a TensorFlowLite tensor (StatefulPartitionedCall:0) with shape [1, 6300, 6] to a Java object with shape [1, 10, 4].
        at org.tensorflow.lite.Tensor.throwIfDstShapeIsIncompatible(Tensor.java:482)
        at org.tensorflow.lite.Tensor.copyTo(Tensor.java:252)
        at org.tensorflow.lite.NativeInterpreterWrapper.run(NativeInterpreterWrapper.java:175)
        at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:360)
        at bline.detector.com.ffmpegvideoeditor.activity.objectDetectorClass.recognizeVideo(objectDetectorClass.java:118)
        at bline.detector.com.ffmpegvideoeditor.activity.PreviewImageActivity.onCreate(PreviewImageActivity.java:118)
        at android.app.Activity.performCreate(Activity.java:7957)
        at android.app.Activity.performCreate(Activity.java:7946)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1307)
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3598)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3775) 
        at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:83) 
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135) 
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95) 
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2261) 
        at android.os.Handler.dispatchMessage(Handler.java:107) 
        at android.os.Looper.loop(Looper.java:237) 
        at android.app.ActivityThread.main(ActivityThread.java:8107) 
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496) 
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100) ```

So what is the error in the logcat? Why do you think the android app that you used for the yolo.tflite will work out of the box? — Farmaker, Jan 18 '22 at 14:19
@Farmaker I have edited the post which includes the logcat error. I forgot to mention that I used the same model trained on my laptop using a pre-trained `SSD MobileNet V2 FPNLite 320x320` on CPU following instructions from this [youtube tutorial series](https://www.youtube.com/watch?v=ZPFcSN6SP7s&t=962s) which the previous student did. I replicated the steps and was able to train the same tflite file but it crashes after conversion. When I test the `.tflite` file on `detect.py` it works, but just doesn't work on Android Studio. Edit: I apologize in advance as I have no knowledge in Java — NYProject, Jan 19 '22 at 02:16
It is a classic error when using an tensorflow example project for a custom model. The error point out that you are trying to feed the model with an array of 1, 6300,6 BUT the model expects 1,10,4. I do not know how you are going to debug that if you have no knowledge of Java. — Farmaker, Jan 19 '22 at 04:46
@Farmaker So it is going to be difficult since I do not have any knowledge in Java and tedious since it involves debugging on Netron? I managed to find people who experience the same problem but they mention modifying the layers on Netron and something about detector inputs. May I know how to obtain these inputs like [this](https://stackoverflow.com/questions/65072546/cannot-copy-from-a-tensorflowlite-tensor-identity-with-shape-1-4-to-a-java)? — NYProject, Jan 19 '22 at 06:21
what does netron.app shows for your tflite file? Is it correct? Can you share a link to that .tflite model to see? — Farmaker, Jan 19 '22 at 06:54
@Farmaker [Here](https://drive.google.com/file/d/1J2tuHP6W-U81cXcD-GTZRQJgJUvjDIlC/view?usp=sharing) is the tflite file. Apparently running on Netron, the first layer starts off with [1,320,320,3] and after `Concentration` there is only one output of the [YOLOV5.tflite](https://drive.google.com/file/d/1qeEo-v6w7BiomfCG7ygcNeYRuagki8JA/view?usp=sharing). However, the [model](https://shorturl.at/moBC7) the previous student used with lower confidence level has 4 outputs and hers is working. Edit: I suppose I need to get the final output to 4 arrays as seen [here](https://shorturl.at/mDEOT)? — NYProject, Jan 19 '22 at 08:18
Yes. I saw that file and it has one output of 1, 6300, 6...but the code expects 1.10.4 ...you need to find out what is going on with your model or with the conversion to tflite — Farmaker, Jan 19 '22 at 09:09

score 0 · Answer 1 · answered Jan 20 '22 at 06:10

0

Your model output has shape [1, 6300, 6] not [1, 10, 4]. You will need to know what is the output of your model means and adjust the code correctly.

answered Jan 20 '22 at 06:10

Karim Nosseir

540
2
8

TensorFlow Lite model not working in Android (Java) but works on Python

1 Answers1