How to record video that is already streamed on onPreviewFrame

Question

I develop an android app which streams video over TokBox. I want to record the streaming video. In order to do this, I tried to use MediaRecorder sample. It did great job on video recording, however I lost my stream. There are two main java classes, just say A and B. The class B implements PreviewCallback. Here are onPreviewFrame method. If you are interested in TokBox, the class B extends BaseVideoCapturer.

@Override
public void onPreviewFrame(byte[] data, Camera camera) {
    mPreviewBufferLock.lock();
    if (isCaptureRunning) {
        if (data.length == mExpectedFrameSize) {
            // Get the rotation of the camera
            int currentRotation = compensateCameraRotation(mCurrentDisplay
                    .getRotation());
            // Send frame to OpenTok
            provideByteArrayFrame(data, NV21, mCaptureWidth,
                    mCaptureHeight, currentRotation, isFrontCamera());

            // Reuse the video buffer
            camera.addCallbackBuffer(data);
        }
    }
    mPreviewBufferLock.unlock();
}

The class A is an activity that manages recording. There is a method to start video record and stop after 5 seconds.

public static boolean prepareAndStartMediaRecorder(){
    if(CustomVideoCapturer.isCaptureStarted){
        // BEGIN_INCLUDE (configure_media_recorder)
        mMediaRecorder = new MediaRecorder();
        // Step 1: Unlock and set camera to MediaRecorder
        B.mCamera.unlock();
        mediaRecorder.setCamera(B.mCamera);
        // Step 2: Set sources
        mediaRecorder.setAudioSource(MediaRecorder.AudioSource.CAMCORDER);
        mediaRecorder.setVideoSource(MediaRecorder.VideoSource.CAMERA);
        // Step 3: Set a CamcorderProfile (requires API Level 8 or higher)
        mMediaRecorder.setProfile(profile);
        // Step 4: Set output file
        mediaRecorder.setOutputFile("/sdcard/myvideo.mp4");
        mediaRecorder.setMaxDuration(60000); // Set max duration 60 sec.
        mediaRecorder.setMaxFileSize(5000000); // Set max file size 5M
        //configure preview
        mediaRecorder.setPreviewDisplay(mPreview.getSurfaceTexture());
        // Step 5: Prepare configured MediaRecorder
        try {
            mediaRecorder.prepare();
        } catch (IllegalStateException e) {
            releaseMediaRecorder();
            return false;
        } catch (IOException e) {
            releaseMediaRecorder();
            return false;
        }
        mediaRecorder.start();
        final Handler handler = new Handler();
        handler.postDelayed(new Runnable() {
            @Override
            public void run() {
                mediaRecorder.stop();  // stop the recording
            }
        }, 5000);
    }

After recording is started, onPreviewFrame method is not called. Do you have any solution about this problem or another approach to record and stream video at the same time?

Edit 1: I tried to apply this solution, but it did not work. If you claim that this is the right solution, please inform me.

Edit 2: The archiving API records the streaming media, so it has noisy sometimes. I need to record frames from camera directly in order to get high-quality video.

did you find a solution to use `MediaRecorder` and `onPreviewFrame` at the time? — user924, Sep 13 '18 at 06:36

HyLian · Answer 1 · 2018-07-24T10:09:46.847

1

If you are using OpenTok and want to record the frame from the camera, the easiest approach is to build a Custom Capturer.

When using OpenTok, a Custom Capturer is a class that fetches frames from anywhere (the camera for example) and feeds them to OpenTok to be streamed. Once you have the frames, you can record them and send them as an OpenTok stream. For more details see the developer guide about custom capturers.

Please take a look at the custom video driver sample for more details.

Here is the code where you can get the frame and create a recording by using something like MediaCodec class.

edited Jul 24 '18 at 10:09

answered Jul 20 '18 at 13:58

HyLian

4,999
5
33
40

I already tried this solution. OpenTok and MediaRecorder need an instance from `Camera` class as a `source`. However, the problem is that the instance from `Camera` class can only be used as a single `source` – Rıdvan Sırma Jul 23 '18 at 06:30
You are right, you cannot use MediaRecorder, I meant MediaCodec instead: https://developer.android.com/reference/android/media/MediaCodec – HyLian Jul 24 '18 at 10:10
1

Are you sure that it can be done with `MediaCodec`? Or is it just a guess? – Rıdvan Sırma Jul 25 '18 at 11:30
Please read the documentation I linked, `MediaCodec` class can be used to encode frames and get a video output – HyLian Jul 25 '18 at 11:35

score 0 · Answer 2 · answered Jul 06 '18 at 10:16

0

OpenTok QA staff here,

If you want to record the content of the session, including audio and video from all participants, you can use the archiving API that OpenTok provides.

https://tokbox.com/developer/rest/#start_archive

answered Jul 06 '18 at 10:16

Francisco Javier Cano Sandoval

294
1
5

That's a good idea. I already tried it, but video recorded via archiving API does not have enough quality. The archiving API records the streaming media, so it has noisy sometimes. I need to record frames from camera directly. – Rıdvan Sırma Jul 07 '18 at 11:18

How to record video that is already streamed on onPreviewFrame

2 Answers2