I want to develop a machine learning program using Tensorflow, OpenCV and Android. I already trained my model and uploaded it to my mobile, but I am facing problems when I want to feed the image taken with camera. The problem is that the camera is taking pictures with the format RGBa (which has 4 channels) and my CNN input has only 3 channels (the shape looks like this 100, 100, 3). Also I want to develop the app that I will make inference on each frame, without the necessity to press a button or something like that. I should say that I am a newbie in Android development and this app would be only to provide a demo to my bachelor thesis.
My network input:
X = tf.placeholder(tf.float32, shape=[1, height, width, channels], name="X")
where height = width = 100 and channels = 3
and my android code looks something like this:
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
javaCameraView = (JavaCameraView)findViewById(R.id.java_camera_view);
javaCameraView.setVisibility(SurfaceView.VISIBLE);
javaCameraView.setCvCameraViewListener(this);
inferenceInterface = new TensorFlowInferenceInterface(getAssets(), MODEL_FILE);
}
@Override
public Mat onCameraFrame(Mat inputFrame) {
mRgba = inputFrame;
Mat resizeimage = new Mat(100, 100, CvType.CV_32SC4);
Imgproc.resize( inputFrame.clone(), resizeimage, new Size(100, 100));
MatOfInt rgb = new MatOfInt(100, 100, CvType.CV_32SC3);
resizeimage.convertTo(rgb,CvType.CV_32SC3);
float[] rgbFloat = new float[(int)(rgb.total()*rgb.channels())];
rgb.get(0,0,rgbFloat);
inferenceInterface.feed(INPUT_NODE, rgbFloat, INPUT_SIZE);
inferenceInterface.run(new String[] {OUTPUT_NODE});
int size = classes.length;
float resu[] = new float[size];
for(int i =0;i< size;i++){
resu[i] = 0;
}
inferenceInterface.fetch(OUTPUT_NODE, resu);
Log.d("result", convert(resu));
return mRgba;
}
How should I proceed?
