1

If I use the Pre-Trained TFLite Object detection model in MLKit, I get the following error:

 CalculatorGraph::Run() failed in Run: 
    Calculator::Open() for node "BoxClassifierCalculator" failed: #vk Unexpected number of dimensions for output index 0: got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1).

Any Idea what I might be doing wrong?

JosephGK
  • 11
  • 1
  • 2

3 Answers3

5

ML Kit does not currently support custom object detection model yet. ML Kit currently only allows developers to use custom image classification models. All TFLite models that are compatible with ML Kit are listed here:

https://tfhub.dev/ml-kit/collections/image-classification/1

If you want to do object detection, you can try out ML Kit's Object Detection API: https://developers.google.com/ml-kit/vision/object-detection

If you want to use a custom object detection model, you can try TFLite task library:

https://www.tensorflow.org/lite/inference_with_metadata/task_library/overview.

Dong Chen
  • 829
  • 4
  • 7
  • How near in the future will custom tflite model be supported for mlkit object detection api? – Jama Mohamed Oct 14 '20 at 16:22
  • 1
    I'm having the same problem, I'm using https://developers.google.com/ml-kit/vision/object-detection/custom-models/android that it's suppose to support custom models. Any ideas? – venimania Feb 18 '21 at 12:56
  • It supports compatible custom classification models listed here: https://tfhub.dev/ml-kit/collections/image-classification/1, but not custom detection models, or other arbitrary custom models. – Dong Chen Feb 21 '21 at 21:41
  • There is no near-term plan for supporting custom detection model. – Dong Chen Feb 21 '21 at 21:43
  • Then what this custom model detection do? https://developers.google.com/ml-kit/vision/object-detection/custom-models/android – lazydevpro Mar 19 '21 at 11:47
  • ML Kit's Object Detection involves two kinds of models: models for object detection and models for image classification. ML Kit supports customization for the latter (with a custom image classification model), but not yet the former (with a custom object detection model). – Dong Chen Mar 28 '21 at 20:40
  • I'm facing the exact same problem, and there is a tiny little note in blue at this page https://developers.google.com/ml-kit/custom-models#model_compatibility_requirements that actually tells about this lack of compability. Amazing – Luis Oct 28 '21 at 12:36
1

Well, since I found out through this post here on SO, I went on to look for other options. And the tutorial posted here: Tensor Flow Lite OBJ detection did the trick just nicely.

First of all added the

implementation 'org.tensorflow:tensorflow-lite-task-vision:0.2.0'

To build.gradle, and this simple code worked like a charm. Obviously I have to make some changes to reflect my specific needs, but it's detecting without errors.

val image = TensorImage.fromBitmap(bitmap)
val options = ObjectDetector.ObjectDetectorOptions.builder()
    .setMaxResults(5)
    .setScoreThreshold(0.5f)
    .build()
val detector = ObjectDetector.createFromFileAndOptions(
    this, // the application context
    "model.tflite", // must be same as the filename in assets folder
    options
)

A more in depth explanation is given in that link, too much to be thrown here. Hope this helps somebody else.

Luis
  • 123
  • 1
  • 3
  • 18
0

You can now create MLKit Custom Models with Google's Vertex AI/AutoML Cloud service and it works well with the CameraX API using an ImageAnalysis Analyzer. Follow the guide here to make and train the TFLite model with your own images via browser (or use the Google Cloud CLI if you have many to upload). After that, this guide walks you through pretty much everything you need to add the model to your project, and how detect and label objects with MLKit.

Alex Conner
  • 240
  • 5
  • 19