Flipping openGL texture from Bytebuffer before creating the Bitmap in Android

Question

I'm using a live streaming API that relies on Google's Grafika repo. I'm using Grafika EGLSurfaceBase's saveFrame method to allow the user to capture stills of his video while he streams.

https://github.com/google/grafika/blob/master/src/com/android/grafika/gles/EglSurfaceBase.java

The actual capture works BUT obviously on some camera orientations the image is flipped.

I've found a lot of questions related to inverted bitmaps taken from OpenGL texture - but most seem to refer to drawn images and rely on either:

a)flipping the texture in OpenG. But in my case, I'm working off of a live streaming API so flipping the texture to capture the image may actually flip the image capture as well on the video stream.

OR

b) flipping the bitmap after it has been generated based on a resource. In my case I don't have a resource,I'm creating the bitmap from the bytebuffer and would rather not duplicate it to flip it.

Here is the basic EGLSurfaceBase Method the API has - I will be passing the camera orientation to it but my question is:

        String filename = file.toString();

    int width = getWidth();
    int height = getHeight();
    ByteBuffer buf = ByteBuffer.allocateDirect(width * height * 4);
    buf.order(ByteOrder.LITTLE_ENDIAN);
    GLES20.glReadPixels(0, 0, width, height,
            GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, buf);
    GlUtil.checkGlError("glReadPixels");
    buf.rewind();

    BufferedOutputStream bos = null;
    try {
        bos = new BufferedOutputStream(new FileOutputStream(filename));
        Bitmap bmp = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
        bmp.copyPixelsFromBuffer(buf);
        bmp.compress(Bitmap.CompressFormat.PNG, 90, bos);
        bmp.recycle();
    } finally {
        if (bos != null) bos.close();
    }
    Log.d(TAG, "Saved " + width + "x" + height + " frame as '" + filename + "'");
}

My preferred solution would be to find a way to flip the image prior to BMP.createbitmap (or at the same time). For example, can I use a matrix to flip the reading of the pixels by glReadPixels?

Another note/thought: maybe the cost of flipping the bitmap after creation is trivial because, since this relies on user interaction, it won't happen often enough to cause a memory error?

score 7 · Answer 1 · answered Jul 10 '15 at 08:02

You can reverse the ByteBuffer after glReadPixels. It is very fast because it is just memory copy. My test showed that reverse operation took less than 10ms.

This is a implementation which is OK:

    private void reverseBuf(ByteBuffer buf, int width, int height)
{
    long ts = System.currentTimeMillis();
    int i = 0;
    byte[] tmp = new byte[width * 4];
    while (i++ < height / 2)
    {
        buf.get(tmp);
        System.arraycopy(buf.array(), buf.limit() - buf.position(), buf.array(), buf.position() - width * 4, width * 4);
        System.arraycopy(tmp, 0, buf.array(), buf.limit() - buf.position(), width * 4);
    }
    buf.rewind();
    Log.d(TAG, "reverseBuf took " + (System.currentTimeMillis() - ts) + "ms");
}

Thank you, dear dragonfly! This code is now being used in YoWindow Weather app to export screen for sharing. — Pavel, Oct 12 '16 at 18:06

score 3 · Answer 2 · answered Feb 06 '15 at 12:41

3

The image always seems flipped when using read pixels because the first pixel in the openGL presented buffer is in bottom left. There are two ways of getting the correct order.

One is to draw it to the buffer upside down which can be done on a separate buffer and will not interfere with your current drawing pipeline at all. This may actually be a pretty good idea as specially if you want to do it on a separate thread or maybe resize the image. This can all be done in a single draw call.

The other is to flip the data manually which is not as bad as it seems since you need to flip the rows only (columns would be pretty bad). Anyway you can actually do this on the same buffer, you do not need a copy of the buffer. Simply keep swapping the lines in order such as: Save the first line, replace it with the last line, replace the last line with the saved one... continue with the second line then.

answered Feb 06 '15 at 12:41

Matic Oblak

16,318
3
24
43

Thanks a lot for the help. I have to admit I'm totally lost on OpenGL between the OpenGL documentation on one hand, and the specificities of OpenGL ES. How do I start on option 1 (draw to buffer upside down)? If you have just a link, or a couple of lines of code I can refer to, it'll help me get off Google and back to the code. – Laurent Feb 06 '15 at 17:07
Btw my solution until your answer came was to create bitmap #2 and copy the first bitmap to it with a transformation matrix (before saving the data to disk). This didn't affect performance much either but I focused on time rather than say, memory allocation. – Laurent Feb 06 '15 at 17:21
Actually this is pretty simple, what you need to do is generate a new frame buffer and the render buffer(s) and attach them as you did with the main buffer. After that, set the correct viewport after binding the frame buffer and you can simply call the drawing code as you did in your main buffer. The only difference is you invert your top and bottom in either your ortho or frustum matrix. Once done simply bind the previous frame buffer to continue. To get more detail search for drawing onto FBO or ask a new question pleas. – Matic Oblak Feb 06 '15 at 20:06

Flipping openGL texture from Bytebuffer before creating the Bitmap in Android

2 Answers2