Using FBO to record screen in Android

Question

I'm developing the Android plugin that record Unity gameplay screen.To achieve this, I used OpenGL FBO. Pseudo Code is so simple like this below :

// Bind frame buffer as a render target
mFrameBuffer.bind();

// Render scene to frame buffer
renderScene();

// Restore rendering target, unbind FBO
mFrameBuffer.unbind();

// Draw texture into display
mTexture.draw(mFrameBuffer.getTexture());

// Make video surface a rendering target
videoCapture.captureFrame(mFrameBuffer.getTexture());

The output video is ok but i had a problem when render to window display using offscreen texture that get from FBO. I checked my code and recognize that the calls to glDrawArray fails with GL_INVALID_OPERATION error code
.

My render code

   public void draw(int textureId) {
    this.shader.use();

    GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
    Log.getPossibleGLError("active texture0");
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureId);
    Log.getPossibleGLError("bind texture when redraw");

    int uOrientationM = this.shader.getAttributeLocation("uOrientationM");
    Log.getPossibleGLError("get orientation matrix");
    int uTransformM = this.shader.getAttributeLocation("uTransformM");
    Log.getPossibleGLError("get transform matrix");

    GLES20.glUniformMatrix4fv(uOrientationM, 1, false, this.orientationMatrix, 0);

    Log.getPossibleGLError("set orientation matrix when redraw");

    GLES20.glUniformMatrix4fv(uTransformM, 1, false, this.transformMatrix, 0);

    Log.getPossibleGLError("set uniform matrix when redraw");

    renderQuad(this.shader.getAttributeLocation("aPosition"));

    Log.getPossibleGLError("render quad when redraw");

    this.shader.unUse();
   }

   private void renderQuad(int aPosition) {
      GLES20.glVertexAttribPointer(aPosition, 2, GLES20.GL_BYTE, false, 0, this.fullQuadVertices);
      Log.getPossibleGLError("get vertex attrib pointer");
      GLES20.glEnableVertexAttribArray(aPosition);
      Log.getPossibleGLError("enable vertex array");
      GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
      // error occurs after execute glDrawArrays
      Log.getPossibleGLError("draw array ");
   }

This code worked fine when render to videoSurface but failed when render to window display.
Any solutions for my issue?

Render to display log : GL_INVALID_OPERATION error occurs

glUseProgram(program = 31)
glActiveTexture(texture = GL_TEXTURE0)
glGetError(void) = (GLenum) GL_NO_ERROR
glBindTexture(target = GL_TEXTURE_2D, texture = 13)
glGetError(void) = (GLenum) GL_NO_ERROR
glGetError(void) = (GLenum) GL_NO_ERROR
glGetError(void) = (GLenum) GL_NO_ERROR
glUniformMatrix4fv(location = 0, count = 1, transpose = false, value = [1.0, 0.0, 0.0, 0.0, -0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0])
glGetError(void) = (GLenum) GL_NO_ERROR
glUniformMatrix4fv(location = 1, count = 1, transpose = false, value = [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0])
glGetError(void) = (GLenum) GL_NO_ERROR
glVertexAttribPointer(indx = 0, size = 2, type = GL_BYTE, normalized = false, stride = 0, ptr = 0x42b1ae38)
glGetError(void) = (GLenum) GL_NO_ERROR
glEnableVertexAttribArray(index = 0)
glGetError(void) = (GLenum) GL_NO_ERROR
glValidateProgram(program = 31)
glGetError(void) = (GLenum) GL_NO_ERROR
glGetProgramiv(program = 31, pname = GL_VALIDATE_STATUS, params = [1])
glGetProgramiv(program = 31, pname = GL_LINK_STATUS, params = [1])
glDrawArrays(mode = GL_TRIANGLE_STRIP, first = 0, count = 4)
glGetError(void) = (GLenum) GL_INVALID_OPERATION
glGetError(void) = (GLenum) GL_NO_ERROR

Render to video log : run fine with no error

glUseProgram(program = 31)
glActiveTexture(texture = GL_TEXTURE0)
glGetError(void) = (GLenum) GL_NO_ERROR
glBindTexture(target = GL_TEXTURE_2D, texture = 13)
glGetError(void) = (GLenum) GL_NO_ERROR
glGetAttribLocation(program = 31, name = uOrientationM) = (GLint) -1
glGetError(void) = (GLenum) GL_NO_ERROR
glGetUniformLocation(program = 31, name = uOrientationM) = (GLint) 0
glGetError(void) = (GLenum) GL_NO_ERROR
glGetError(void) = (GLenum) GL_NO_ERROR
glGetAttribLocation(program = 31, name = uTransformM) = (GLint) -1
glGetError(void) = (GLenum) GL_NO_ERROR
glGetUniformLocation(program = 31, name = uTransformM) = (GLint) 1
glGetError(void) = (GLenum) GL_NO_ERROR
glGetError(void) = (GLenum) GL_NO_ERROR
glUniformMatrix4fv(location = 0, count = 1, transpose = false, value = [1.0, 0.0, 0.0, 0.0, -0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0])
glGetError(void) = (GLenum) GL_NO_ERROR
glUniformMatrix4fv(location = 1, count = 1, transpose = false, value = [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0])
glGetError(void) = (GLenum) GL_NO_ERROR
glGetAttribLocation(program = 31, name = aPosition) = (GLint) 0
glGetError(void) = (GLenum) GL_NO_ERROR
glVertexAttribPointer(indx = 0, size = 2, type = GL_BYTE, normalized = false, stride = 0, ptr = 0x42b1ae38)
glGetError(void) = (GLenum) GL_NO_ERROR
glEnableVertexAttribArray(index = 0)
glGetError(void) = (GLenum) GL_NO_ERROR
glValidateProgram(program = 31)
glGetError(void) = (GLenum) GL_NO_ERROR
glGetProgramiv(program = 31, pname = GL_VALIDATE_STATUS, params = [1])
glGetProgramiv(program = 31, pname = GL_LINK_STATUS, params = [1])
glVertexAttribPointerData(indx = 0, size = 2, type = GL_BYTE, normalized = false, stride = 2, ptr = 0x??, minIndex = 0, maxIndex = 4)
glDrawArrays(mode = GL_TRIANGLE_STRIP, first = 0, count = 4)
glGetError(void) = (GLenum) GL_NO_ERROR
glGetError(void) = (GLenum) GL_NO_ERROR
glUseProgram(program = 0)

Failed as in what? The render display, the video, errors...? — Matic Oblak, Aug 01 '14 at 06:41
Failed in render display. I always get GL_INVALID_OPERATION after execute glDrawArrays command — knighthedspi, Aug 01 '14 at 06:49
Something is probably missing or incorrect. Try checking the frame buffer state after you create it. You didn't post any code but if you are attaching a texture to that buffer it might need to be POT(power of two dimensions). — Matic Oblak, Aug 01 '14 at 07:07
I checked the frame buffer state after creating.If i failed to create frame buffer, I would failed to get offscreen texture. I don't think this is the reason causes this error because i can get the offscreen texture and use it to render to videoSurface — knighthedspi, Aug 01 '14 at 07:32
I see your edit now, the video is OK... Could you test if this is the texture issue: Try drawing without the texture (all the code left the same) just to see if the error persists. — Matic Oblak, Aug 01 '14 at 07:43
I tried drawing without texture and got the same error:GL_INVALID_OPERATION after execute glDrawArrays command! — knighthedspi, Aug 01 '14 at 09:20
The only condition I found in the spec where `glDrawArrays()` would give `GL_INVALID_OPERATION` is if the shader program cannot be executed with the current state. Can you try using `glValidateProgram()` before the draw call, and see if it succeeds? — Reto Koradi, Aug 01 '14 at 09:39
@MaticOblak: If it were a problem with the framebuffer, the error should be `GL_INVALID_FRAMEBUFFER_OPERATION`, not `GL_INVALID_OPERATION`. — Reto Koradi, Aug 01 '14 at 09:40
I tried using glValidateProgram() before call glDrawArrays but no errors occurred.GL_INVALID_OPERATION only occurs after execute glDrawArrays command — knighthedspi, Aug 01 '14 at 10:06
Is your main frame buffer bound when you call draw? You seem to only unbind the FBO which probably results in no frame buffer bound at all. — Matic Oblak, Aug 01 '14 at 10:08
Yes, your main frame buffer which is used to push the data to the display. The frame buffer to which you are trying to draw the texture beside the FBO. — Matic Oblak, Aug 01 '14 at 10:12
I only unbind my own FBO.Unbind it means the default FBO provided by window system will be bound.Isn't it the main FBO which is used to push the data to the display??? — knighthedspi, Aug 01 '14 at 10:23
I would most definitely not be counting on that. EVERY frame buffer has an ID which is assigned by the GPU so the only way this was done behind your back was if the framework uses a push/pop stack system where the main buffer is the bottom most item. But I highly doubt it. Also there is no such thing as default buffer or buffer provider by the window system, there are usually only convenience methods to create the buffer from the window parameter. Also you can have multiple views each having its own frame buffer... — Matic Oblak, Aug 01 '14 at 10:27
According to OpenGL doc https://www.khronos.org/opengles/sdk/docs/man/xhtml/glBindFramebuffer.xml, Framebuffer object names are unsigned integers. The value zero is reserved to represent the default framebuffer provided by the windowing system. Maybe i should try to get FBO provided by Unity? — knighthedspi, Aug 01 '14 at 10:35
I think this should be interpreted as the 0 ID buffer is reserved and you may not use it at all. You are working with view buffers, not with window buffer. And yes, pleas try to use the one provided by unity. — Matic Oblak, Aug 01 '14 at 10:38
Can i get this value by using glGetIntegerv with first parameter is GL_FRAMEBUFFER_BINDING? — knighthedspi, Aug 01 '14 at 11:07
I believe so. Try it by putting that before binding the FBO and then bind the buffer with ID received by this method after you unbind the FBO. — Matic Oblak, Aug 01 '14 at 11:24
I tried getting FBO that provided by unity and the ID received is 0.And the same error GL_INVALID_OPERATION still occurs whenever I try to render to window display — knighthedspi, Aug 02 '14 at 02:31
I updated OpenGL trace log to my question? Any solution for this issue? Please help!!! — knighthedspi, Aug 06 '14 at 09:13

score 5 · Accepted Answer · answered Aug 27 '14 at 02:28

5

Finally after many investigations about OpenGl, I fixed this issue. Problem is when I attempted to redraw texture to screen, the array buffer and element array buffer were wrongly pointed.Solution is so simple, I created my own array buffer and element array buffer then bind them before attempt to redraw to screen. It worked fine with no errors :D

answered Aug 27 '14 at 02:28

knighthedspi

526
5
17

Can you please share the working code for this? Trying to do the same thing during the last week but stack with the same problem - glDrawArray generates GL_INVALID_OPERATION error while using android plugin in unity. Already investigating an option to move to NDK with the example available here: http://docs.unity3d.com/Manual/NativePluginInterface.html but it doesn't work for android for some reason (though works for mac). Thanks in advance! – vir us Nov 07 '15 at 17:46
1

I uploaded my code to this https://gist.github.com/knighthedspi/2b7d7b29a7ad8401f415 Hope it can help you !!! – knighthedspi Nov 09 '15 at 06:31
Thank you very much! At this moment I've ended up with GLUtils.texImage2D() approach which successfully renders a bitmap into unity texture on java side. I'll give your code a try anyway since texImage2D is not very good at constant rendering in terms of performance. – vir us Nov 11 '15 at 20:13

Using FBO to record screen in Android

1 Answers1