Pause/Resume MediaCodec

Question

I'm trying to implement pause/resume feature for my app that records display capture using MediaCodec. I've tried doing mEncoder.stop() and then mEncoder.start() without calling mEncoder.release() but that didn't work. I get IllegalStateException when calling mEncoder.start() again. Right now I implemented a workaround, I'm merging peaces of video after the capture is complete, but it takes a really long time to merge. Can anyone help me with that? Maybe someone has already implemented this thing?

Initialazation:

    MediaCodec mEncoder;
    mEncoder = MediaCodec.createEncoderByType(Preferences.MIME_TYPE);
    mEncoder.configure(mFormat, null, null,
            MediaCodec.CONFIGURE_FLAG_ENCODE);
    mInputSurface = new InputSurface(mEncoder.createInputSurface(),
            mSavedEglContext);
    mEncoder.start();
    try {
        String fileId = String.valueOf(System.currentTimeMillis());
        mMuxer = new MediaMuxer(dir.getPath() + "/Video"
                + fileId + ".mp4",
                MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
        videoParts.add(fileId);
    } catch (IOException ioe) {
        throw new RuntimeException("MediaMuxer creation failed", ioe);
    }
    isRecording = true;

Pause:

    public void pauseRecordPressed() {
    if (isRecording){
        isRecording = false;
        drainEncoder(false);

        if (mEncoder != null) {
            mEncoder.stop();
        }
    }
}

Unpause:

    public void resumeRecordPressed() {
    mEncoder.start();
    isRecording = true;
}

Exception:

01-09 15:34:27.980: E/AndroidRuntime(21467): FATAL EXCEPTION: main
01-09 15:34:27.980: E/AndroidRuntime(21467): Process: com.example.poc, PID: 21467
01-09 15:34:27.980: E/AndroidRuntime(21467): java.lang.IllegalStateException: start failed
01-09 15:34:27.980: E/AndroidRuntime(21467):    at android.media.MediaCodec.start(Native Method)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at com.example.poc.MyRenderer.resumeRecordPressed(MyRenderer.java:501)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at com.example.poc.MyGLSurfaceView.resumeRecordPressed(MyGLSurfaceView.java:243)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at com.example.poc.MainActivity.onClick(MainActivity.java:775)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at android.view.View.performClick(View.java:4438)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at android.view.View$PerformClick.run(View.java:18422)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at android.os.Handler.handleCallback(Handler.java:733)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at android.os.Handler.dispatchMessage(Handler.java:95)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at android.os.Looper.loop(Looper.java:136)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at android.app.ActivityThread.main(ActivityThread.java:5017)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at java.lang.reflect.Method.invokeNative(Native Method)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at java.lang.reflect.Method.invoke(Method.java:515)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:779)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:595)
01-09 15:34:27.980: E/AndroidRuntime(21467):    at dalvik.system.NativeStart.main(Native Method)

MediaFormat:

mFormat = createMediaFormat();

    private static MediaFormat createMediaFormat() {
    MediaFormat format = MediaFormat.createVideoFormat(
            Preferences.MIME_TYPE, mScreenWidth, mScreenHeight);
    format.setInteger(MediaFormat.KEY_COLOR_FORMAT,
            MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface);
    format.setInteger(MediaFormat.KEY_BIT_RATE, Preferences.BIT_RATE);
    format.setInteger(MediaFormat.KEY_FRAME_RATE, Preferences.FRAME_RATE);
    format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL,
            Preferences.IFRAME_INTERVAL);
    return format;
}

Hi Alex, I am also facing same issue. It would be very helpful to me, if you share solution(code). — abhishek kumar gupta, Oct 17 '14 at 10:46

fadden · Accepted Answer · 2014-01-10T15:49:59.330

10

The MediaCodec discards its configuration when stopped, so you will need to call configure() again. I'm not sure why you're trying to do restart it though -- you can just leave it active, without feeding it data.

For example, see the CameraCaptureActivity in Grafika, which leaves the encoder alive across activity restarts. If you don't want to have a pause in the video during the restart, you'll need to keep track of how long encoding was paused, and then adjust the timestamps being fed into the muxer.

edited Jan 10 '14 at 15:49

answered Jan 09 '14 at 15:57

fadden

51,356
5
116
166

I'm doing so because of this peace of information:http://developer.android.com/reference/android/media/MediaCodec.html#start(). stop() Finish the decode/encode session, note that the codec instance remains active and ready to be start()ed again. But thanks for the suggestion, I'll try that! – Alexey Jan 10 '14 at 07:14
Could you please give me some example on how to adjust timestampes for MediaCodec video capture. This would help me a lot. Thanks in advance! – Alexey Jan 10 '14 at 08:18
I didn't think there was a CTS test that exercised this (which usually equates to "it doesn't work"), but it turns out http://bigflake.com/mediacodec/#DecoderTest does. You just need to call `configure()` again [answer updated]. For the timestamps, record the current time in `onPause()` and `onResume()` to figure out how long you were asleep, then subtract that from every timestamp you submit from then on. Make sure you get the time scale right (microseconds vs. milli- or nano-) or crazy things can happen. – fadden Jan 10 '14 at 15:50
Fadded, thank you very much for your suggestions. I did what you suggested and now I'm facing the issue with timestamps, I'm not sure I understand correctly how they work. If you could please take a look at my question: http://stackoverflow.com/questions/21094782/unable-to-mux-both-audio-and-video. This would help me a lot – Alexey Jan 13 '14 at 15:19
1

fadden, thank you very much! I did exactly what you suggested and I was able to remove the gap between video parts. Now I only have one black frame in place of merge. I don't beleive it's possible to get rid of it though? – Alexey Jan 14 '14 at 13:21
To be clear, are you doing stop/configure/start, or leaving it active? I haven't tried the former, so I don't know if a black frame is expected. – fadden Jan 14 '14 at 15:34
I do stop/configure/start – Alexey Jan 15 '14 at 07:10
Okay... are you by any chance writing the `CODEC_CONFIG` frame into the stream? That's always the first thing out of the encoder after it starts. – fadden Jan 15 '14 at 15:30
1

Yep, that was the problem. I forgot to reset frame counter after pause/resume. Thanks again! – Alexey Jan 16 '14 at 05:55
Hi fadden, thanks for the "Grafika". I am having one query that how we can adjust the timestamps being fed into the muxer? I also want to remove the pause in the video. Please help me. – abhishek kumar gupta Oct 23 '14 at 05:35
The timestamps are passed to the muxer by the application, so "adjust the timestamps" just means the app changes the values being passed through. You can track the pause time by capturing the current time in onPause / onResume and subtracting the duration from the PTS values. – fadden Oct 23 '14 at 15:26
Hi fadden, thanks for the reply. I did what you suggested and now I'm facing one issue that while resuming, recorded video is getting distorted for 1-2 seconds at the resume instant(means video is unclear for 1-2 seconds and after that it is perfect) . I have tried to adjust(subtract some value) timestamp which I fed to MediaMuxar but not able to get rid-off from distortion(only able to reduce it). Any suggestion will be very helpful to me. – abhishek kumar gupta Oct 27 '14 at 05:58
Hi fadden, I am able to reduce the distortion in video while resuming by decreasing I-Frame interval but not able to completely remove the distortion. Any suggestion or advice will be very helpful to me. – abhishek kumar gupta Oct 30 '14 at 05:57
@abhishek kumar gupta This is a long time ago but did you figure your problem out? I am having the exact same issue. – Furedal Jan 03 '20 at 10:06

score 3 · Answer 2 · answered May 18 '19 at 06:01

private void suspendMediaCodec(boolean suspend)
{
    if(mediaCodec != null)
    {
        Bundle params = new Bundle();
        params.putInt(MediaCodec.PARAMETER_KEY_SUSPEND, suspend ? 1 : 0);
        mediaCodec.setParameters(params);
    }
}

You can suspend/resume mediacodec to drop frames by setting parameter MediaCodec.PARAMETER_KEY_SUSPEND. Above one is example to do it.

Pause/Resume MediaCodec

2 Answers2