0

I'm using the Android NDK MediaCodec API to decode an MP4 stream of varying resolutions.

I am using the built-in functionality to render to a surface, calling AMediaCodec_releaseOutputBuffer with the render flag set to true.

I have found that every time the resolution changes, several frames of the old resolution are output on a surface the size of the new resolution. It might look something like this for a step up in resolution:

+------------------+-------------+
| frame of old res |             |
| displayed too    |             |
| small            |             |
+------------------+             |
|                                |
|      size of new resolution    |
+--------------------------------+

After a few frames, the video looks normal again. I suspect it's something to do with the surface changing in size when an input buffer is queued with a new resolution, rather than when the first output buffer of that resolution is dequeued.

Has anyone encountered this issue before?

jww
  • 97,681
  • 90
  • 411
  • 885
Jack
  • 2,153
  • 5
  • 28
  • 43
  • Interestingly this problem never reared it's head on my ADT-1 device, only on my new Nexus Player. – Jack Jan 28 '15 at 18:11
  • Is your decoder configured to work in `metadata-on-output-buffers` mode or in general mode? When a resolution changes, the decoder will try to flush out the pending frames in it's queue. So if the allocated buffer is big enough to handle the current resolution, you should get a `portsettingschanged` event in the framework. Can you specify which version of `android` you are working on and share any captured logs in `ACodec`? – Ganesh Jan 29 '15 at 00:39

2 Answers2

2

Thanks for your help.

I found the following in the MediaCodec documentation:

For some video formats it is also possible to change the picture size mid-stream. To do this for H.264, the new Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) values must be packaged together with an Instantaneous Decoder Refresh (IDR) frame in a single buffer, which then can be enqueued as a regular input buffer. The client will receive an INFO_OUTPUT_FORMAT_CHANGED return value from dequeueOutputBuffer() or onOutputBufferAvailable() just after the picture-size change takes place and before any frames with the new size have been returned.

Whenever I receive a new SPS and PPS I make sure not to pass them to the decoder straight away, but rather to prepend them to the next I-Frame.

This solved the problem - resolution changes are seamless :)

Jack
  • 2,153
  • 5
  • 28
  • 43
0

Does this particular decoder indicate support for MediaCodecInfo.CodecCapabilities.FEATURE_AdaptivePlayback?

I haven't encountered this issue myself (since I haven't tried decoding streams that change resolution), but as far as I know, this feature flag was introduced to signal proper handling for this use case. If you see this behaviour on a decoder with this feature flag, I would consider it a bug (or I haven't understood the actual implications of this feature flag properly).

mstorsjo
  • 12,983
  • 2
  • 39
  • 62
  • `Adaptive Playback` was introduced to support `DASH` which can have resolution change at ay point of time. Typically, decoders will be able to handle resolution change by signalling a port settings changed or updated events based on the strategy. A decoder not supporting `adaptive playback` flag can still decide to decode a stream if the new resolution is less than or equal to old resolution and signal a cropping window change or alternatively request for reallocation through port settings changed. – Ganesh Jan 29 '15 at 00:42