Use webrtc VideoCapturer without peerconnection?

Question

Is it possible to use webrtc VideoCapturer without peerconnection?

We have a working androidapp app (from examples/androidapp). We have taken following code from the working app into a separate activity where we use camera capturer directly without creating peerconnection. We create a video capturer (camera2) using an instance of CapturerObserver and then try to render it to org.webrtc.SurfaceViewRenderer. Below is the code.

As expected, onFrameCaptured of the CapturerObserver is being called multiple times with valid videoFrame object. From there, we pass it to SurfaceViewRenderer. However, video does not render and SurfaceViewRenderer remains black.

Is that a correct way of using VideoCapturer and SurfaceViewRenderer? Does it require any format conversion before sending to SurfaceViewRenderer?

private class MyCapturerObserver implements CapturerObserver {
    @Override
    public void onCapturerStarted(boolean b) {
        Log.e(TAG, "capture started: " + b);
    }

    @Override
    public void onCapturerStopped() {
        Log.e(TAG, "capture stopped");
    }

    @Override
    public void onFrameCaptured(final VideoFrame videoFrame) {
        //fullscreenRenderer.onFrame(videoFrame);
        runOnUiThread(new Runnable() {
            @Override
            public void run() {
                fullscreenRenderer.onFrame(videoFrame);
            }
        });

    }
}

capturer = createVideoCapturer();
captureObserver = new MyCapturerObserver();
surfaceTextureHelper =
            SurfaceTextureHelper.create("CaptureThread", eglBase.getEglBaseContext());

capturer.initialize(surfaceTextureHelper, getApplicationContext(), captureObserver);
capturer.startCapture(1280, 720, 30);

I see you have the same problem as me Because you do runOnUiThread() https://stackoverflow.com/questions/53441496/can-a-texture-be-updated-in-background-thread-but-printed-in-foreground-thread and is it possible to know how you handle it ? — zeus, Nov 23 '18 at 06:23

score 4 · Accepted Answer · edited Nov 05 '18 at 22:35

Use factory.createVideoSource. You can use it before creating peerconnection. You can refer source code in the PeerConnectionClient.java

public VideoTrack createVideoTrack(VideoCapturer capturer) {
    surfaceTextureHelper = SurfaceTextureHelper.create("CaptureThread", rootEglBase.getEglBaseContext());
    videoSource = factory.createVideoSource(capturer.isScreencast());
    capturer.initialize(surfaceTextureHelper, appContext, videoSource.getCapturerObserver());
    capturer.startCapture(videoWidth, videoHeight, videoFps);

    localVideoTrack = factory.createVideoTrack(VIDEO_TRACK_ID, videoSource);
    localVideoTrack.setEnabled(renderVideo);
    localVideoTrack.addSink(localRender);
    return localVideoTrack;
}

Hii I have some kind of issue with this, I have changed from Video renderer to add sink but all over process become two flow. and I still don't get what SurhfaceTextureHelper does and how to behave .can anyone help me with this? — Richa Shah, Apr 10 '20 at 06:16

Use webrtc VideoCapturer without peerconnection?

1 Answers1