4

I really need to get an RGB 8 bytes per channel buffer from the GPU. I need it to pass to a trained convolutional neural network, and it only accepts data in that format. I can't convert it on the CPU as I'm heavily CPU bound and it's quite slow.

I currently have FBO with a renderbuffer attached, which is defined with: glRenderbufferStorage(GL_RENDERBUFFER, GL_RGB8_OES, bufferWidth, bufferHeight); There are no errors when I bind, define and render to the buffer.

But when I use glReadPixels(0, 0, bufferWidth, bufferHeight, GL_RGB, GL_UNSIGNED_BYTE, rgbBufferRawName); it gives an invalid enum error (0x0500). It works just fine when I pass GL_RED_EXT or GL_RGBA and produces correct buffers (I've checked it by uploading those buffers to a texture and rendering them, and they looked correct).

I tried setting glPixelStorei(GL_PACK_ALIGNMENT, 1); but that made no difference.

I'm on iOS10 and iPhone 6. I was doing ES2.0, but now tried switching to ES3.0 in hopes that it will help me solve the problem. It did not.

I would really appreciate help in getting RGB8 buffer in any way, Thanks.

grisevg
  • 250
  • 3
  • 18
  • I would suggest that instead of a FBO with render buffer you use a FBO with a color texture attachment (and depth attachment if you need a depth buffer), then use glGetTextureImage to read the data from it. glGetTextureImage accepts GL_RGB format while glReadPixels does not. – VB_overflow Dec 09 '16 at 17:10
  • @VB_overflow `glGetTextureImage` is not available on OpenGL ES 3.0 =( – grisevg Dec 09 '16 at 19:23
  • My bad, sorry. In this case I would suggest using a PBO like described [here](https://vec.io/posts/faster-alternatives-to-glreadpixels-and-glteximage2d-in-opengl-es). With this method you get a ptr to the RGBA pixel data, so instead of doing a memcpy like in the example you could do a loop which will only copy 3 bytes (RGB), skip one (A) and repeat... This will not be as fast as a memcpy but probably pretty close. Combined with some double or triple buffering this should allow you to have a decent execution speed in the end. – VB_overflow Dec 12 '16 at 09:57
  • @VB_overflow Yeah, it's worth switching FBO to PBO regardless as there's way less CPU overhead. Do you think it's worth implementing RGBA -> RGB with SIMDs? Or it's purely bandwidth bound and will make no difference? – grisevg Dec 12 '16 at 11:59
  • I am not an expert in SIMD on IOS (Neon instruction set) but at first sight I would say that you would not gain much from those for simple RGBA to RGB "strided memcopy" as the compiler will probably already optimize the loop pretty well (you compile in 64bit right ?). – VB_overflow Dec 12 '16 at 15:09

1 Answers1

1

According the OpenGL 3.0 specification, GL_RGB is not a valid value for format.

https://www.khronos.org/opengles/sdk/docs/man3/html/glReadPixels.xhtml

You may want to either convert it to RGB after retrieving the GL_RGBA formatted buffer, or adjusting your algorithm to compensate for RGBA.

Stephen
  • 317
  • 1
  • 11
  • So, there's no way to do it on the GPU? – grisevg Dec 16 '16 at 11:28
  • I'm unsure of how your algorithm works, but if you are aiming to do it entirely on the GPU using GLSL, you could do this as two passes. In the first pass, you can render the scene to a texture, and in the second pass run your algorithm. The output could be to the screen and use glReadPixels to read the values, or you can write to a uniform variable and read it from the shader after it runs. – Stephen Dec 16 '16 at 13:52
  • I was talking about getting GL_RGB8 buffer... Maybe if I switch to Metal there will be a way. It's really silly that on iOS exotic thing like GL_RED works fine, but GL_RGB doesn't... But I guess my use case is quite unique. I think I'll just have to hack neural network to accept RGBA and skip over alpha, but it's going to be quite ugly and hacky – grisevg Dec 16 '16 at 15:33
  • It sounds like you are looking for a CPU solution (glReadPixels or equivalent) and not a GPU solution (shaders). If the format of the pixels is the issue, I would suggest looking into OpenCV. You could get the buffer using GL_RGBA, and quickly and easily convert it to an RGB image. Good luck! – Stephen Dec 16 '16 at 16:22