7

I wish to encode video/avc on my android encoder. The encoder (Samsung S5) publishes COLOR_FormatYUV420Flexible as one of its supported formats. yay!

but I dont quite understand what it is and how I can use it. the docs say:

Flexible 12 bits per pixel, subsampled YUV color format with 8-bit chroma and luma components.

Chroma planes are subsampled by 2 both horizontally and vertically. Use this format with Image. This format corresponds to YUV_420_888, and can represent the COLOR_FormatYUV411Planar, COLOR_FormatYUV411PackedPlanar, COLOR_FormatYUV420Planar, COLOR_FormatYUV420PackedPlanar, COLOR_FormatYUV420SemiPlanar and COLOR_FormatYUV420PackedSemiPlanar formats

This seems to suggest that I can use this const with just about any kind of YUV data: planer, semi-planer, packed etc. this seems unlikely: how would the encoder know how to interpret the data unless I specify exactly where the U/V values are?

is there any meta-data that I need to provide in addition to this const? does it just work?

Community
  • 1
  • 1
FuzzyAmi
  • 7,543
  • 6
  • 45
  • 79

1 Answers1

10

Almost, but not quite.

This constant can be used with almost any form of YUV (both planar, semiplanar, packed and all that). But, the catch is, it's not you who can choose the layout and the encoder has to support it - it's the other way around. The encoder will choose the surface layout and will describe it via the flexible description, and you need to support it, whichever one it happens to be.

In practice, when using this, you don't call getInputBuffers() or getInputBuffer(int index), you call getInputImage(int index), which returns an Image, which contains pointers to the start of the three planes, and their row and pixel strides.

Note - when calling queueInputBuffer afterwards, you have to supply a size parameter, which can be tricky to figure out - see https://stackoverflow.com/a/35403738/3115956 for more details on that.

Community
  • 1
  • 1
mstorsjo
  • 12,983
  • 2
  • 39
  • 62
  • Thank you for answering. So if I understood correctly, this const is only useful when calling getInputImage() - which is not my use case - since I need to feed the encoder from a buffer. – FuzzyAmi Jul 18 '16 at 05:43
  • 1
    Yes, but even when feeding the encoder from a buffer, it is certainly useful. When not using this, you choose one of the supported color formats, and convert/relayout from your input data into the encoder's input buffer while copying. When using Image, you do pretty much exactly the same, except you don't really know the exact layout of the encoder's input buffer, you only have the generic Image description of it, so which exact layout it is (planar, semiplanar, etc), you only know once you have the Image. – mstorsjo Jul 18 '16 at 05:46
  • 1
    So instead of having one ByteBuffer with a layout that is known beforehand, you get three ByteBuffers (which actually in practice point into the same single buffer which is the encoder's input buffer) and a description of the strides. The main difference is that the copy/relayout routine needs to be a bit more flexible. – mstorsjo Jul 18 '16 at 05:47
  • Wow. Thanks for clarifying this: I've been reading the relevant docs for a week now and managed to miss that entirely. – FuzzyAmi Jul 18 '16 at 05:59