8

The official definition for presentationTimeUs in queueInputBuffer (int index, int offset, int size, long presentationTimeUs, int flags) is the following:

The presentation timestamp in microseconds for this buffer. This is normally the media time at which this buffer should be presented (rendered).

Why does the decoder need this if it is up to the app when to present the decoded image? I have tried some arbitrary numbers for presentationTimeUs, and they do not seem to make any difference to the decoding. For example, if I double the original values of presentationTimeUs, the video seems to be decoded exactly the same way and at the same speed as the original.

Could anyone shed some light on this?

Community
  • 1
  • 1
Hong
  • 17,643
  • 21
  • 81
  • 142

1 Answers1

2

The decoder needs to know the input buffer's timestamps for multiple reasons.

First, if the stream has B-frames, then the reordering of buffers and assigning the correct timestamps to the buffers is performed by the decoder. So when the timestamps are received on the input buffer, the same is queued for reordering.

Secondly, if the use-case is something like Android-TV which infact has the tunneled video playback, the timestamp is consumed by the video decoder which is tunneled to the underlying HW block for synchronization and rendering.

Lastly, if there is any drop in packets or frames, the decoder can potentially perform some sort of concealment if it an observes abrupt jump in timestamps without a flush being called. This is not a norm, but is an advanced feature of some decoders.

In traditional cases, as you have pointed out, the synchronization is performed by the player engine in which decoder shall reflect the input buffer timestamp onto the output buffer.

Ganesh
  • 5,880
  • 2
  • 36
  • 54
  • 1
    Thanks a lot for the answer. It makes a lot sense. The presentationTimeUs of MediaCodec.BufferInfo returned by dequeueOutputBuffer should be respected. – Hong May 31 '15 at 10:29
  • 2
    Hi, what does `presentationTimeUs` mean when *I am encoding from raw data (pcm/wav) to aac*? Thanks! – ch271828n Sep 17 '20 at 02:41
  • In the uplink / encoding path, you are encoding continuous data. Hence, time should increase linearly for frames. For ex: If frame size is 20 ms, the time stamp of frames should be 0, 20, 40, 60 ... When it's decoded, this shall be the 'presentationTimeUs' which will be used for rendering.. Hope this helps – Ganesh Sep 18 '20 at 05:14