Picture produced by saveFrame of Grafika is upside-down

Question

I wrote a demo according to Continuous capture activity.I wanted to save the first frame of the video as a jpeg file, so I used the saveFrame() function which was already prepared by Grafika. The source code of the function is as follows: https://github.com/google/grafika/blob/master/src/com/android/grafika/gles/EglSurfaceBase.java

But I found that the jpeg picture produced was upside-down.This is not what I want.

Is there a way of rotating the frame before I save the frame?

Rotating the bitmap is not a graceful solution,because we might encounter some other situations such as video chatting. Each frame of the video is sent to the remote peer, then the remote person will see a upside-up video. So we should rotate every frame before it is encoded.

score 1 · Accepted Answer · answered Jul 10 '15 at 03:20

You don't want to rotate the frame, you want to flip the frame.

The situation exists because Bitmap thinks (0,0) is in the top-left corner, but GLES thinks (0,0) is in the bottom-left corner. glReadPixels() starts filling the buffer from the top, which Bitmap thinks is the bottom, so you end up with an upside-down image.

You have two options:

Draw the frame upside-down with GLES, then grab it as you are now. This is what the ExtractMpegFramesTest code does -- note the drawFrame() function takes an invert argument, and modifies the matrix obtained from the SurfaceTexture if it's set.
Draw and extract the frame as you are now, then flip the pixels in the buffer before converting them to a Bitmap. This is a simple matter of moving bytes around. It will add some overhead, but you're already kind of slow because of the JPEG compression. (Copying whole lines with System.arrayCopy() might be better than copying individual bytes with Java code.)

I chose option 2 because the first frame will also be encoded to the video stream. I don't want the first frame to be upside-down in the video. — dragonfly, Jul 10 '15 at 08:09

score 1 · Answer 2 · answered Jul 10 '15 at 08:10

1

My implementation is as follows which is very fast, less than 10ms.

    private void reverseBuf(ByteBuffer buf, int width, int height)
{
    long ts = System.currentTimeMillis();
    int i = 0;
    byte[] tmp = new byte[width * 4];
    while (i++ < height / 2)
    {
        buf.get(tmp);
        System.arraycopy(buf.array(), buf.limit() - buf.position(), buf.array(), buf.position() - width * 4, width * 4);
        System.arraycopy(tmp, 0, buf.array(), buf.limit() - buf.position(), width * 4);
    }
    buf.rewind();
    Log.d(TAG, "reverseBuf took " + (System.currentTimeMillis() - ts) + "ms");
}

answered Jul 10 '15 at 08:10

dragonfly

1,151
14
35

where do you put it? – Maverick Meerkat Feb 19 '19 at 12:22
I assume you use it before calling mMuxer.writeSampleData... Do you know if there's a way to reverse a direct ByteBuffer ? – Maverick Meerkat Feb 19 '19 at 13:56
I use this method to capture video frame to jpg. – dragonfly Feb 22 '19 at 07:12
For video encoding, we should process every frame, this method is too slow. We should use the GPU. The GPU way is that render the frame to a new fbo with a rotation matrix, so image in the new fbo will has the right direction. Then we read the image to memory and give it to the h264 encoder. – dragonfly Feb 22 '19 at 07:13

Picture produced by saveFrame of Grafika is upside-down

2 Answers2

Linked