Android MediaCodec encoding + camera2 how to create a black screen for pause and resume

Question

I have created a surface with the MediaCodec and added it as a target to the camera2 api. Now I want to sometime pause the MediaCodec encoding and then resume it. What I expect is the encoded video to become black when I pause and continue when I resume. Using MediaCodec.PARAMETER_KEY_SUSPEND doesn't do the job since the output is a frozen frame, and if the paused happens at the beginning of encoding the time starts when I resume. So I was thinking of feeding the encoder with black image, but how can I draw on the surface? Thanks!

Edit: Here is some of the code I use to make the camera2 work with MediaCodec.

I create a simple MediaCodec:

try {
    mediaCodec =
        MediaCodec.createEncoderByType(mime)
    Logger.d(TAG, "MediaCodec info " + mediaCodec.codecInfo.name)


} catch (e: IOException) {
    e.printStackTrace()
}
mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)

val inputSurface = mediaCodec.createInputSurface() //<-- this is passed to the camera2

I add the surface to the camera:

captureRequestBuilder.addTarget(inputSurface);
List<Surface> surfaceList = new ArrayList<>();
        surfaceList.add(cameraFrame.getSurface());
        surfaceList.add(inputSurface);

        camera.createCaptureSession(
                surfaceList,
                captureSessionObserver,
                cameraThreadHandler
        );

And then in a loop I read the mediaCodec output buffers and write them into a muxer.

easy_breezy · Answer 1 · 2021-12-25T17:05:52.737

1

I would draw a black image on a surfaceView something like this:

val surface = getSurfaceFromSomething()
val canvas = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) surface.lockHardwareCanvas()
else surface.lockCanvas(null)
canvas.drawColor(Color.BLACK)
surface.unlockCanvasAndPost(canvas)

Or also you can try this MediaCodec feature: https://developer.android.com/reference/android/media/MediaFormat#KEY_PUSH_BLANK_BUFFERS_ON_STOP

edited Dec 25 '21 at 17:05

answered Dec 25 '21 at 16:42

easy_breezy

1,073
7
18

Tested the KEY_PUSH_BLANK_BUFFERS_ON_STOP - did not work. When do you think is the time to do the surface thing? – Uriel Frankel Dec 25 '21 at 18:50
You can try to draw on a surface right after you called some of mediaCodec methods to pause it. If it won't work you can try to detect BUFFER_FLAG_END_OF_STREAM and draw a black frame right after you got this flag. Also there are a couple of dirty hacks you can just set black background color to the SurfaceView which the surface belong to (surfaceView.setBackgroundColor(Color.BLACK)) or you can create an ImageView with black background and place it under surfaceView on the screen and hide it when you resume mediaCodec and show it when you pause. – easy_breezy Dec 25 '21 at 19:37
This sounds like hacks that might work on 1 device but not for others. – Uriel Frankel Dec 25 '21 at 21:19
As for the first two things yeah it might take place, but as for the last two ones I would like to note that they will work on all the devices – easy_breezy Dec 25 '21 at 21:37
I tried the surfaceview.setBackgroundColor and it didn't do anything. the camera2 has list of surface that it write into. – Uriel Frankel Dec 25 '21 at 21:56
What I want to acheive is that the recording will be black, not the view. – Uriel Frankel Dec 25 '21 at 22:06
Your answer isn't working.. – Uriel Frankel Dec 26 '21 at 14:36
can you share some code how you create your surface set it to camera 2 api and how you tried to draw a black frame on the surface? And another question why are you using Camera 2 and not CameraX ? – easy_breezy Dec 27 '21 at 15:24
I will share code, but this is why i use camera2 and not camerax https://stackoverflow.com/a/56303135/588532 – Uriel Frankel Dec 27 '21 at 15:29

Android MediaCodec encoding + camera2 how to create a black screen for pause and resume

1 Answers1