How to integrate yolo-v3 custom object detector in flutter app?

Question

I developed flutter app and yolov3 custom object detector. Both modules are independent. Now I want to combine those modules into a single project but could not figure out how to use that trained weights of that custom object detector in my flutter app. Could anyone please help me with this integration?

Answer 1

I don't know whether using Flutter you're building android app or iOS.

Anyway to be able to use custom trained Yolov3 model on your Flutter app, follow these two steps.

1. First you need to convert trained yolov3 model to tflite version:

You can use this repo for that purpose.

Save custom trained Yolov3 darknet weights to tfmodel that's needed for tflite conversion:

python save_model.py --weights yolov3.weights --output ./checkpoints/yolov3-416 --input_size 416 --model yolov3 --framework tflite

Convert Yolov3 model to tflite version:

python convert_tflite.py --weights ./checkpoints/yolov3-416 --output ./checkpoints/yolov3-416.tflite

2. Then you use Flutter plugin for accessing TensorFlow-Lite API, which works with both android and iOS - https://github.com/shaqian/flutter_tflite

a) Create a assets folder and place your label file and model file in it. In pubspec.yaml add:

  assets:
   assets/labels.txt
   assets/yolov3-416.tflite

b) Import the library:

import 'package:tflite/tflite.dart';

c) Load the model and labels:

String res = await Tflite.loadModel(
model: "assets/yolov3-416.tflite",
labels: "assets/labels.txt",
numThreads: 1, // defaults to 1
isAsset: true, // defaults to true, set to false to load resources outside assets
useGpuDelegate: false // defaults to false, set to true to use GPU delegate
        );

d) To run on image:

  var recognitions = await Tflite.detectObjectOnImage(
  path: filepath,       // required
  model: "YOLOv3",      
  imageMean: 0.0,       
  imageStd: 255.0,      
  threshold: 0.3,       // defaults to 0.1
  numResultsPerClass: 2,// defaults to 5
  anchors: anchors,     // defaults to [0.57273,0.677385,1.87446,2.06253,3.33843,5.47434,7.88282,3.52778,9.77052,9.16828]
  blockSize: 32,        // defaults to 32
  numBoxesPerBlock: 5,  // defaults to 5
  asynch: true          // defaults to true
);

e) Release resources:

await Tflite.close();