Grafika TextureMovieEncoder

Question

I have recently been modifying Grafika's TextureMovieEncoder to create a recording of what I displayed onscreen: two Sprite2ds which are overlapping. Using the CameraCaptureActivity example as a reference point, I effectively ported what I created for my rendering thread into the TextureMovieEncoder but the output is jagged lines across the screen. I think I understand what's wrong, but I don't know how to fix it:

Some code:

private void prepareEncoder(EGLContext sharedContext, int width, int height, int bitRate,
        File outputFile) {
    try {
        mVideoEncoder = new VideoEncoderCore(width, height, bitRate, outputFile);
    } catch (IOException ioe) {
        throw new RuntimeException(ioe);
    }
    mEglCore = new EglCore(sharedContext, EglCore.FLAG_RECORDABLE);
    mInputWindowSurface = new WindowSurface(mEglCore, mVideoEncoder.getInputSurface(), true);
    mInputWindowSurface.makeCurrent();

    textureProgram = new Texture2dProgram(Texture2dProgram.ProgramType.TEXTURE_EXT);

    backgroundDrawable = new Drawable2d(Drawable2d.Prefab.RECTANGLE);
    backgroundRect = new Sprite2d(backgroundDrawable);
    frontDrawable = new Drawable2d(Drawable2d.Prefab.RECTANGLE);
    frontRect = new Sprite2d(frontDrawable);

    backgroundRect.setTexture(backTextureId);
    frontRect.setTexture(frontTextureId);

    updateGeometry();
}

private void handleFrameAvailable(Transform transform, long timestampNanos) {
    if (VERBOSE) Log.d(TAG, "handleFrameAvailable tr=" + transform);
    mVideoEncoder.drainEncoder(false);

    backgroundRect.draw(textureProgram, transform.movieMatrix);
    frontRect.draw(textureProgram, transform.cameraMatrix);

    mInputWindowSurface.setPresentationTime(timestampNanos);
    mInputWindowSurface.swapBuffers();
}

I think the problem comes down to my lack of understanding of how to establish the right projection onto the WindowSurface for the VideoEncoder. In the Grafika example, FullFrameRect is used, which is easier since you can just use the identity matrix to stretch a given texture to the surface area. However, since I want to create the overlapping effect, I needed to use Sprite2d. Is the problem the shared EGLContext? Do I need to create a new one so that I can set the viewport to match the WindowSurface size? A bit lost on where to go from here.

In many ways the "continuous capture" activity is a better starting point, as it avoids the GLSurfaceView-induced gymnastics; in particular it uses a single EGL context. In any event, the EGL contexts share textures and other goodies but not the viewport or any of the drawing state, so all that gets configured independently. Do you see bad output on-screen only, on video only, or in both? The "texture from camera" activity shows feeding video (from the camera) into a Sprite2d rather than a FullFrameRect. — fadden, Nov 11 '14 at 16:46
Hey fadden. Thanks for the clarification. I see output from both, but the video is messed up with aliasing. I tried setting the viewport and then using orthM to setup the projection matrix, but that didn't work (the video has still aliasing...but differently). `GLES20.glViewport(0, 0, width, height);` `Matrix.orthoM(mDisplayProjectionMatrix, 0, 0, width, 0, height, -1, 1);` — kungfoo, Nov 11 '14 at 20:04
Also. If I change all the drawing to use FullFrameRect instead of Sprite2d, the output video shows up without aliasing; however, without the correct scaling/positioning. — kungfoo, Nov 11 '14 at 20:10
That seems fine. Is it a problem with geometry or with texturing? i.e. are you seeing the shapes with the correct positions and size, but a video texture rendered onto it is messed up? cf. the "hardware scaler exerciser", which flips between flat-shaded and textured shapes. — fadden, Nov 12 '14 at 01:03
Hey fadden, you were right on the money with this. The problem was with the initialization of my Sprite2d. I was passing the texture ids too early or too late. Before the thread had spun up and also after the recording had started. As such, it was just pulling wrong textures — kungfoo, Nov 15 '14 at 01:16
Hey fadden, kungfoo I am trying create a video recorder using Grafika. I have created a TextureView and creating a encoder using Circular encoder. If video is played uninterrupted the new video on video completion is fine. But if the video is paused in between then the new video has the still frames from time the video was paused. can you please help in this? — Ravneet Singh, Aug 03 '17 at 13:34

score 1 · Accepted Answer · answered Nov 15 '14 at 01:18

Turns out the functionality of the code above was fine. The problem was the interaction between the TextureEncoder and the calling parent.

I was initializing the member variables backTextureId and frontTextureId after prepareEncoder and it was therefore recording garbage data into the output.

Grafika TextureMovieEncoder

1 Answers1