1

I'm trying to stream video in android through ffmpeg,the output which i am getting after the decoding is YUV format.Is it possible to render YUV image format directly in the Android screen?

fadden
  • 51,356
  • 5
  • 116
  • 166
vgokul129
  • 777
  • 12
  • 31

1 Answers1

1

Yes and no.

The output of the camera and hardware video decoders is generally YUV. Frames from these sources are generally sent directly to the display. They may be converted by the driver, typically with a hardware scaler and format converter. This is necessary for efficiency.

There isn't an API to allow an app to pass YUV frames around the same way. The basic problem is that "YUV" covers a lot of ground. The buffer format used by the video decoder may be a proprietary internal format that the various hardware modules can process efficiently; for your app to create a surface in this format, it would have to perform a conversion, and you're right back where you were performance-wise.

You should be able to use GLES2 shaders to do the conversion for you on the way to the display, but I don't have a pointer to code that demonstrates this.

Update: an answer to this question has a link to a WebRTC source file that demonstrates doing the YUV conversion in a GLES2 shader.

Community
  • 1
  • 1
fadden
  • 51,356
  • 5
  • 116
  • 166
  • Thanks for your replay,actually i am trying this with GLES1.Can you help me with what i have proceed with GLES1,now i have my decoeded frames ready to render the screen – vgokul129 Oct 13 '14 at 15:25
  • Shaders don't exist until GLES2. There might be a way with GLES1 but I don't know what it is. – fadden Oct 13 '14 at 15:48
  • As there are many YUV formats exists,can you know anything related to these formats,which exactly android supports? – vgokul129 Oct 13 '14 at 15:52
  • The problem is that the YUV format used by the Surface is often proprietary. See http://bigflake.com/mediacodec/#q5 for some notes on MediaCodec output. API 19 added ImageReader to access YUV directly, but that only works with Camera output. There is no equivalent for MediaCodec output, and no equivalent for input. And if your target API is high enough to have access to these classes, you might be better off using MediaCodec anyway. – fadden Oct 13 '14 at 17:20
  • :Thanks for the link,it was quite useful.What i'm doing here was converting the decoded YUV frames to RGB through SWS_SCALE() and tried rendering the screen,but the screen remains black. – vgokul129 Oct 14 '14 at 06:21