Frame and keras model

Question

I have a model which predicting the steering angle of a car from a picture which I like to implement in a android project by getting the frames from the camera.

in my Python code I'm using a h5 file instead of the tflite file in which the picture is converted to numpy and get processed using Cv2 lib.

Python Code:

import numpy as np
import tensorflow as tf
from tensorflow.keras.models import load_model
from PIL import Image
import cv2


def img_preprocess(img):
    img = img[60:135,:,:]
    img = cv2.cvtColor(img, cv2.COLOR_RGB2YUV)
    img = cv2.GaussianBlur(img,  (3, 3), 0)
    img = cv2.resize(img, (200, 66))
    img = img/255
    return img


if __name__ == '__main__':
    model = load_model('model.h5')
    image1 = Image.open("Gta2.png")

    image1 = np.asarray(image1)
    image1 = img_preprocess(image1)
    image1 = np.array([image1])
    steering_angle = float(model.predict(image1))
    if(steering_angle > 0):
        print('turn right')
        print('turn wheel : {}'.format(steering_angle))

    else:
        print('turn left')
        print('turn wheel : {}'.format(steering_angle))

I've been imported the model.tflite to my project assets and know I need to process the CameraBridgeViewBase.CvCameraViewFrame object from the camera to fit to my model input.

the model input and output.

so my questions are:

1. How process the CvCameraViewFrame Object as in the python code in the 'img_preprocess' function?
2. How to reach the input specs?

my android code:

Answer 1

You can first convert your Mat frame to Bitmap, such as the solution in this question.

And then use TFLite Task library ImageClassifier to run the inference. Here is an example of how to use ImageClassifier, which comes from the TFLite Image Classification reference app.

Alternatively, you can write your own code to process the image and run inference using Interpreter. See the example here.