1

I am trying to build a react-native android app for augmented reality purposes which I need a live camera preview displayed on the screen. So I'm trying to make my own Android camera component instead of using React Native's default react-native-camera package. This is because I need a camera module that would allow me to get camera frames in real time, detect Aruco markers, draw on the image frame, and display it on a camera preview in react-native. So from reading tutorials and articles, I've attempted to create a view manager class and a ReactPackage class to bridge my Android camera view. FYI the CameraBridgeViewBase class is an OpenCV class which extends SurfaceView, which is found in this library: https://github.com/quickbirdstudios/opencv-android/tree/master/opencv3_4_4_contrib

When I run the app, I just keep getting a black screen with no camera preview. I did have camera permissions granted before the view gets created. If there is any suggestions or anyone who has experience in this area that could help. I would greatly appreciate, Thanks.

View Manager Class:

public class JavaCameraViewManager extends SimpleViewManager<CameraBridgeViewBase> implements CameraBridgeViewBase.CvCameraViewListener2 {

    public static final String REACT_CLASS = "JavaCameraView";
    public CameraBridgeViewBase javaCameraView;

    @Override
    public String getName() {
        return REACT_CLASS;
    }

    @Override
    protected CameraBridgeViewBase createViewInstance(ThemedReactContext reactContext) {
        javaCameraView = new JavaCameraView(reactContext, null);
        javaCameraView.setVisibility(SurfaceView.VISIBLE);
        javaCameraView.setCvCameraViewListener(this);
        return javaCameraView;
    }

    @Override
    public void onCameraViewStarted(int width, int height){}

    @Override
    public void onCameraViewStopped(){}

    @Override
    public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame){
        System.out.println("onCameraFrame$");
        return inputFrame.rgba();
    }

}

React Package class:

public class JavaCameraViewPackage implements ReactPackage {

    @Override
    public List<NativeModule> createNativeModules(ReactApplicationContext reactContext) {
        return Collections.emptyList();
    }
    @Override
    public List<ViewManager> createViewManagers(ReactApplicationContext reactContext) {
        return Collections.<ViewManager>singletonList(
                new JavaCameraViewManager()
        );
    }
}

React Native Camera View Component class: 

import React, { Component } from 'react'
import { requireNativeComponent} from 'react-native';

const JavaCameraView = requireNativeComponent('JavaCameraView', JavaCameraViewView);

export default class JavaCameraViewView extends Component {
    render () {
      return <JavaCameraView {...this.props} />
    }
  }

App.js:

'use strict';
import React, {Component} from 'react';
import {Platform, StyleSheet, Text, View} from 'react-native';
import { NativeModules, AppRegistry, TouchableOpacity} from 'react-native';
import JavaCameraView from './src/JavaCameraViewNativeView'

export default class App extends Component  {

  render() {
    return (
      <View style={styles.container}>
        <JavaCameraView style={{ flex: 1, width: '100%', height: '100%', backgroundColor:'blue'}} /> 
      </View>
    );
  }
}
Adrian
  • 41
  • 7

2 Answers2

1

I found an opencv project that has a solution on how to set up android openCV java camera preview to bridge to react native. Check out https://github.com/RobertSasak/react-native-openalpr

Adrian
  • 41
  • 7
0

Should be able to get this working if you add a couple more lines of code. I have tested this with OpenCV v4.5.4 and v3.4.10 on an expo (SDK 44) bare workflow project. I was running into a few errors with the Impgroc functions in my onCameraFrame function using OpenCV v4.5.4 and have no issues since downgrading to v3.4.10.

// Class should extend SimpleViewManager<JavaCameraView>
public class JavaCameraViewManager extends SimpleViewManager<JavaCameraView> implements CameraBridgeViewBase.CvCameraViewListener2 {
    ...
    // public CameraBridgeViewBase javaCameraView;
    public JavaCameraView javaCameraView;
    ...
    @Override
    public JavaCameraView createViewInstance(ThemedReactContext reactContext) { 
        javaCameraView = new JavaCameraView(reactContext, -1);
        javaCameraView.setCvCameraViewListener(this);
        // javaCameraView.setCameraPermissionGranted(); // Only required on newer versions of OpenCV
        javaCameraView.enableView();
        return javaCameraView;
    }
    ...
}

You should also make sure you have camera permissions granted in AndroidManifest.xml based on your needs

  <!-- Camera & mic & flashlight permissions -->
  <uses-permission android:name="android.permission.CAMERA" />
  <uses-permission android:name="android.permission.RECORD_AUDIO"/>
  <uses-permission android:name="android.permission.FLASHLIGHT" />

Hope this helps anyone struggling to set this up.

Sam
  • 1
  • 1