3

I have put together an app that detects the coordinates of faces and puts a square around them using expo-camera. What I actually need to do is crop all the faces from the camera stream with 1-second interval, and pass them through a TensorFlowjs model with TensorFlow.js For React Native. I couldn't see anything like that in expo-camera documentation, is that possible with some sort of a trick?

Mertcan Seğmen
  • 903
  • 2
  • 11
  • 26

1 Answers1

0

I'm looking into doing something similar and I think these might help.

RNCamera a component with a onModelProcessed prop that can be used to work with the camera's stream. https://www.npmjs.com/package/react-native-tflite-camera

Here's an example I found that might help:

https://medium.com/@namar/high-performance-image-classification-with-react-native-336db0a96cd

OR

cameraWithTensors a component designed by the TF people has "the ability to yield tensors representing the camera stream." Once you have the tensors you can process them or use them as input in a tfjs model.

Here's an example using cameraWithTensors:

https://www.linkedin.com/pulse/real-time-image-classification-tensorflow-daniel-wind/?utm_campaign=React%2BNative%2BNow&utm_medium=web&utm_source=React_Native_Now_69

It sounds like you might need two model, one that detects the faces and returns the coordinates/pixel to crop at. Then after cropping, you'd need to put the cropped images into another model.

Ryan Whetten
  • 1