0

I'm trying to record the contents of an fbo that i create and fill in c++ with the MediaCodec java object.

I understand that i need to get a Surface from MediaCodec and draw on it with a frag shader, so i'm trying to feed the c++ fbo as uniform to the shader.

At the moment i can write the video and if i draw all the pixels red with the shader i got a red clip, so everything ok on the Shader=>Surface=>MediaCodec side.

To use my fbo i'm passing the GLunit of the texture of the fbo to the java side with a jni function, and everything seems ok, but all i got is just noise, like a not initializated texture.

So the question is: is in any way possible to use a texture done with c++ as a input uniform to a gles2 java shader?

and if yes, what is the correct flow to do so?

To clarify, i'm actually doing this in my GLSurfaceView.Renderer class:

@Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
    Log.i("OF","onSurfaceCreated");
    OFAndroid.onSurfaceCreated();
    try{
        ((OFActivity)OFAndroid.getContext()).onGLSurfaceCreated();
    }catch(Exception e){
        Log.e("OF","couldn call onGLSurfaceCreated",e);
    }
    return;
}

@Override
public void onSurfaceChanged(GL10 gl, int w, int h) {
    this.w = w;
    this.h = h;
    if(!setup && OFAndroid.unpackingDone){
        try {
            setup();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    OFGestureListener.swipe_Min_Distance = (int)(Math.max(w, h)*.04);
    OFGestureListener.swipe_Max_Distance = (int)(Math.max(w, h)*.6);
    OFAndroid.resize(w, h);

    mRenderFbo = new RenderFbo(
            800, 800, mSrfTexId);
    mFboTexId = mRenderFbo.getFboTexId();

    GlUtil.checkGlError("onSurfaceCreated_E");

}

@Override
public void onDrawFrame(GL10 gl) {
    if(setup && OFAndroid.unpackingDone){
        OFAndroid.render();

        if (mRenderSrfTex != null) {
            mRenderSrfTex.draw();
        }
        GlUtil.checkGlError("onDrawFrame_E RENDER SRF TEX");
    }else if(!setup && OFAndroid.unpackingDone){
        try {
            setup();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }else{
        gl.glClear(GL10.GL_COLOR_BUFFER_BIT);
        gl.glClearColor(.5f, .5f, .5f, 1.f);    
    }
}

public void setRecorder(MyRecorder recorder, int myTexID) {
    texID = myTexID;
    synchronized(this) {
        if (recorder != null) {
            mRenderSrfTex = new RenderSrfTex(
                    800, 804,
                    myTexID, recorder);
        } else {
            mRenderSrfTex = null;
        }
    }
}

where myTexID is the GLUint that i pass from c++ to java with jni, containing the GLUint of the fbo texture.

for reference to the classes myRecorder and RenderSrfTex see the classes at https://github.com/MorihiroSoft/Android_MediaCodecTest/tree/master/MediaCodecTest18/src/jp/morihirosoft/mediacodectest18

and here the sources of the relevant part of the framework, https://github.com/openframeworks/openFrameworks/blob/master/addons/ofxAndroid/ofAndroidLib/src/cc/openframeworks/OFAndroidWindow.java

rcavazza
  • 11
  • 3
  • The Android Java wrappers are pretty thin, so anything that works in Java should also work in native. All of the state should be in the EGL context, so passing integer indices around should work. You can find a Java-only example in Grafika's "record GL app" activity (https://github.com/google/grafika/blob/master/src/com/android/grafika/RecordFBOActivity.java). It might help to include some of your code in the question. – fadden Jul 18 '15 at 14:53
  • Hi fadden, i've already studied all of your examples for weeks, but without any help, cause you always do everything on the java side. If everything is created from a GLSurfaceView, also the c++ part, does the objects in the onDrawFrame() share the context with the java part or i need to do something? do i have to get an error if i try to access a context that is not shared?? – rcavazza Jul 18 '15 at 17:31
  • GLSurfaceView sometimes makes things more complicated, because it manages its own EGL context. Bear in mind that GLES state is thread-specific, and a context can only be "current" on one thread at a time, so all of your GLES code (C++ and Java) must be running on the GLSurfaceView's renderer thread if you want access to that context. (You can try to create a shared context, but I don't know offhand how that interacts with FBOs, and the "texture from camera" activity shows what a pain it is to set up with GLSurfaceView.) – fadden Jul 18 '15 at 18:05
  • The shader in mediacodectest18/RenderFbo.java is using a `samplerExternalOES`, which is incorrect for an "internal" texture. For an FBO, it should be using a `sampler2D`. (Note RenderSrfTex.java and RenderScreen.java are using `sampler2D`.) – fadden Jul 18 '15 at 20:02

1 Answers1

1

The answer is absolutely yes, it's possible to pass via GLUint a textureID from c++ to Java, and here there is my solution to record openFrameworks ofFbo with MediaCodec https://github.com/rcavazza/openFramework-Android-Recording. Thanks fadden for the useful answers.

rcavazza
  • 11
  • 3