2

We've built an async MediaCodec transcoder that uses a surface generated from a MediaCodec decoder using the newer API 21+ async callbacks (onInputBufferAvailable and onOutputBufferAvailable). It's fully functional, but we've noticed that the decoding / encoding speed is directly tied to the PTS frame timing data. I.e. if the source file is 1 fps, the transcoder works rather slowly, yet on a 60fps file it works much faster. As this is a transcoding design (rather the realtime playback), we're looking to decode / encode as fast as the system can handle, and be entirely independent of the source / destination framerate.

We've managed to work around this issue by using the KEY_OPERATING_RATE MediaFormat key (i.e. setting it to 10x FPS) but this seems to crash MediaCodec when using high ratios, and we are concerned there may be some device specific behavior here. According to the API docs it seems like we should be able to use releaseOutputBuffer() with a timestamp that is 1+ seconds away from the current system time to force the frame to render immediately, but this doesn't seem to have any effect. We've also tried the KEY_LATENCY and KEY_PRIORITY MediaFormat keys, but they also appear to have no effect.

Is this as expected? Is there any way to get the MediaCodec encoder / decoder to output frames immediately (independent of frame timing) when using async mode?

fadden
  • 51,356
  • 5
  • 116
  • 166
Dustin Kerstein
  • 450
  • 5
  • 13
  • Have you checked the GOP structure of the output? IIRC, MediaCodec uses the PTS values to fit the stream into the configured max bitrate. At 1 fps it might be outputting a solid stream of I-frames, which could be larger and slower to handle than a mix of frames. Quick experiment: divide the PTS by 30 when feeding it into MediaCodec, then multiply by 30 before passing it on to MediaMuxer (or wherever it is you're sending the output). (The PTS isn't included in the elementary stream output by MediaCodec.) – fadden Apr 08 '19 at 15:22
  • We're actually always encoding to 100% I Frames with the MediaCodec encoder so I don't believe it's that. Apologies for not making that clear. Also, the source is a long GOP h264 elementary stream (always using the same test file for debug and manually setting the FPS key and PTS data on the frame). Thanks for the suggestion though. Any other things come to mind? – Dustin Kerstein Apr 09 '19 at 09:19

0 Answers0