6

I have an Android app that decodes video into yuv420p format then renders video frames using OpenGLES. I use glTexSubImage2D() to upload y/u/v buffer to GPU then do a YUV2RGB conversion using shader. All EGL/OpenGL setup/rendering code is native code.

Now I am not saying there is no problem with my code, but considering the same code is running perfecting fine on iOS (iPad/iPhone), Nexus 7, Kindle HD 8.9, Samsung Note 1 and a few other cheap chinese tablets (A31/RockChip 3188) running Android 4.0/4.1/4.2. I would say it's less likely my code is wrong. On those devices, glTexSubImage2D() uses less than 16ms to upload a SD or 720P HD texture.

However, on Nexus 10, glTexSubImage2D() it takes about 50~90ms for a SD or 720P HD texture which is way too slow for a 30fps or 60fps video.

I would like to know

1) if I should pick a different texture format (RGBA or BGRA). Is there a ways to detect which is the best texture format used by a GPU?

2) if there is a feature that is 'OFF' on all other SOCs but set to 'ON' on Exynos 5. For example, the automatic MIPMAP generation option. (I have it off, btw)

3) if this is a known issue of Samsung Exynos SOC - I can't find a support forum for Exynos CPU

4) Is there any option I need to set when configure the EGL surface? like, transparency, surface format, etc? (I have no idea what I am talking about)

5) It could mean GPU is doing an implicit format conversion but I checked GL_LUMINANCE is always used. Again it works on all other platform.

6) anything else?

My EGL config:

const EGLint attribs[] = {
    EGL_SURFACE_TYPE, EGL_WINDOW_BIT,
    EGL_RENDERABLE_TYPE, EGL_OPENGL_ES2_BIT,
    EGL_BLUE_SIZE, 8,
    EGL_GREEN_SIZE, 8,
    EGL_RED_SIZE, 8,
    EGL_NONE
};

Initial setup:

glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, ctx->frameW, ctx->frameH, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, NULL); /* also for U/V  */

subsequent partial replacement:

glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, ctx->frameW, ctx->frameH, GL_LUMINANCE, GL_UNSIGNED_BYTE, yBuffer); /*also for U/V */

I am trying to render video at ~30FPS or ~60FPS at SD or 720P HD resolution.

Yi Wang
  • 552
  • 2
  • 7
  • 20
  • Consider the `SurfaceTexture` for Android > 11: http://developer.android.com/reference/android/graphics/SurfaceTexture.html – Jave Mar 22 '13 at 23:05
  • Thanks but I still would like to figure out why glTexSubImage2D() is slow since I don't want to maintain multiple code path - it works for all except Exynos – Yi Wang Mar 22 '13 at 23:07
  • We observed the same (or very similar) issue and reported it at https://code.google.com/p/android/issues/detail?id=53135, but it does seem to be a regression between 4.2.1 and 4.2.2 based on our tests. – chuoling Apr 02 '13 at 21:18

3 Answers3

6

This is a known driver issue that we have reported to ARM. A future update should fix it.

Romain Guy
  • 97,993
  • 18
  • 219
  • 200
1

EDIT Status update

We've now managed to reproduce slow upload conditions for one path on the public firmware, which you are possibly hitting, and this will be fixed in the next driver release.

If you double-buffer texture IDs (e.g. frame N = ID X, N+1 = ID Y, N+2 = ID X, N+3 = ID Y, etc) for the textures you are uploading to it should help avoid this on the current firmware.

Thanks, Iso

solidpixel
  • 10,688
  • 1
  • 20
  • 33
  • Much appreciated. So just to confirm you are uploading a semi-planar YUV surface as two 8-bit luminance textures, one with 8-bit Y, and one with two 4-bit U/V values packed into an 8-bit channel. – solidpixel Mar 23 '13 at 11:29
  • Hmm I don't have enough rep points to comment on Romain's answer, so I'll post it here. No this is not a regression between 4.2 to 4.2.1, so rolling back won't help. – solidpixel Mar 23 '13 at 11:41
  • yes I am uploading a YUV420P (nv12) texture. Is there a workaround? Say, uploading RGB texture instead? – Yi Wang Mar 23 '13 at 13:05
  • Interleaving the Y and the UV planes (so you have "YUVYUVYUV" rather than "YYY", "UVUVUV"), and loading as a LUMINANCE_ALPHA texture perhaps might work. – solidpixel Mar 23 '13 at 18:31
  • ... but as Romain says this is something we are working on, so you might just want to wait for the next version of the driver. – solidpixel Mar 23 '13 at 18:42
  • Any idea when 'the next version of the driver' will be available or integrated into Android? – Yi Wang Apr 05 '13 at 00:36
  • Sorry - can't talk about release plans. As per my edit above we are actually struggling to reproduce this one, so if you can share a test case (source or APK) which reproduces this it would be really useful for us to ensure that it gets fixed. – solidpixel Apr 08 '13 at 18:23
  • OK I tried generating two texture ids(by glGenTextures() ) and use each one (by glBindTexture()) by turn. This does not help improve texture uploading time. Anything else I should check? – Yi Wang Apr 22 '13 at 19:49
  • After a bit more digging I suspect you are actually hitting a different issue; we've diagnosed a general texture upload performance issue for some color formats. We know that 32-bit textures should work more consistently but obviously will increase memory footprint. Fixed in the next firmware drop. – solidpixel Jul 18 '13 at 16:27
  • video usually uses yuv420 color format so not only 32-texture uses more memory than yuv420p (talking about 60fps 720p video), converting all frame from yuv to rgb is expensive too – Yi Wang Jul 30 '13 at 03:27
1

I can confirm this has been fixed in Android 4.3 - I'm seeing a performance increase by a factor of 2-3 with RGBA format and by a factor of 10-50 with other texture formats over Android 4.2.2. These results apply for both glTexImage2D and glTexSubImage2D. (Can't add comments yet so I had to put this here)

EDIT: If you're stuck with 4.2.2, you could try using RGBA texture instead, it should have better performance (3-10x or so with larger power-of-two texture sizes).

Arttu Peltonen
  • 1,250
  • 14
  • 18
  • Apparently, it is still not fast enough. My app started using glTexSubImage2D with an RGBA texture, and from that release onwards it is crashing on Nexus 10. It doesn't crash on any other Android out there! – HRJ Nov 06 '13 at 10:42