2

I am trying to get the information about all the macroblocks in the frames of a video (mp4). In particular i'm using the ffmpeg command:

ffmpeg -debug mb_type -i input.mp4 out.mp4 2> macroblocks.txt

It seems to work fine, but... i don't understand how to parse the output!

I see that after many uninteresting writings, starts a group of blocks starting with

"New frame, type: [FRAME TYPE]"

so I assume that these are the blocks referring to each frame containing the type of each macroblock.. but what do the symbols inside mean?

New frame, type: B [h264 @ 000001c0241c1cc0] d < X- < I > > > > X d d d d d < < d < d > < d d > d < d d d < > < d < > X < d d > d X d < > d X d > > d d+ d

From the theory I know that there are intra or predicted macroblocks, but i don't understand how to parse this info from the "New frame"-blocks.

  • What means i,I,A,<,>,X,|,etc.?

Also often there are sentences like

nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2

or

cur_dts is invalid (this is harmless if it occurs once at the start per stream)

that i really don't understand... I can't too find a documentation.. Can anyone help me?

micha137
  • 1,195
  • 8
  • 22
fabridigua
  • 53
  • 1
  • 7

1 Answers1

5

The best documentation seems to be the source code, currently starting at line 196 of libavcodec/mpegutils.c. I won't duplicate everything here, just enough to understand the example line given above.

Each macroblock is described by 3 characters:

  1. type and motion vector direction

    • d: IS_DIRECT && IS_SKIP
    • <: !USES_LIST(0) - Reference to future (List 1, B slices)
    • X: USES_LIST(0) && USES_LIST(1) - Reference to past and future (List 1 & 2, B slices)
    • >: !USES_LIST(1) - Reference to past (List 0, P or B slices)
    • for more see the code

  2. segmentation

    • +: IS_8X8
    • -: IS_16X8
    • |: IS_8X16
    • space: IS_INTRA || IS_16X16
    • ?: otherwise

  3. interlacing

    • =: IS_INTERLACED
    • space: not interlaced

Also interesting in this connection is the macro block type visualization built into ffmpeg itself.

For the NAL unit types see table 7-1 here.

micha137
  • 1,195
  • 8
  • 22
  • OK i'll read the code. Another question: do you know why in the report returned by ffmpeg, each frame has different number of "characters" associated (is like they had different number of macroblocks, but i belive it should be not possible) – fabridigua Jun 17 '18 at 08:48
  • @fabridigua Some blocks are subdivided into 8x8 or even 4x4 blocks, so may have more characters based on that, maybe? – ndtreviv Dec 16 '19 at 15:27