0

I am using the tflite_flutter package https://pub.dev/packages/tflite_flutter to give a TFlite AI model a set of images from the live camera feed. I'm starting with their example given on their site that shows how to give a single image from the camera to the TFlite model. I'd like to this for all images in a single second of the camera feed, which is about 10 total images (there are in fact 30 images per second but we use every third only). I can put the images in a list like [image0.buffer, image1.buffer,..., imageN-1.buffer] but then I need to convert this list into a single object buffer to give to the TFlite model - the conversion is where I am facing a problem.

To be more concrete, the tflite_flutter package has an example of copying an image buffer of size [300,300,3] to a tensor of size [1, 300, 300, 3] where the first 1 is the batch size. But I want to do this for a list of 10 images, so something like [10, 300, 300, 3] to [1, 10, 300, 300, 3].

The function that copies a single image to a tensor is shown below.

void setTo(Object src) {
    Uint8List bytes = _convertObjectToBytes(src);
    int size = bytes.length;
    final ptr = calloc<Uint8>(size);
    checkState(isNotNull(ptr), message: 'unallocated');
    final externalTypedData = ptr.asTypedList(size);
    externalTypedData.setRange(0, bytes.length, bytes);
    checkState(tfLiteTensorCopyFromBuffer(_tensor, ptr.cast(), bytes.length) ==
        TfLiteStatus.ok);
    calloc.free(ptr);
  }

To copy a list of image buffers we call the above function with src set to a list of 10 image buffers [image0.buffer, image1.buffer,..., image9.buffer]. It gives the error below which doesn't tell us the exact problem.

E/flutter ( 1068): [ERROR:flutter/runtime/dart_isolate.cc(1098)] Unhandled exception:
E/flutter ( 1068): Bad state: failed precondition
E/flutter ( 1068): #0      checkState (package:quiver/check.dart:74:5)
E/flutter ( 1068): #1      Tensor.setTo (package:tflite_flutter/src/tensor.dart:146:5)
E/flutter ( 1068): #2      Interpreter.runForMultipleInputs (package:tflite_flutter/src/interpreter.dart:187:33)
E/flutter ( 1068): #3      Classifier.predict (package:ace_example/tflite/classifier.dart:195:17)
E/flutter ( 1068): #4      IsolateUtils.receiveMsgFromRootIsolate (package:ace_example/tflite/isolate_utils.dart:61:49)
E/flutter ( 1068): <asynchronous suspension>

To put this into a broader context, we want to process a live video stream through our TFlite model and have it send messages to the user on a Flutter mobile app. Any help would be appreciated and I'm happy to provide more info on the problem as needed. Thanks.

Ross
  • 265
  • 1
  • 3
  • 13

0 Answers0