Android ImageView overlay over Opengl ES unexpected behavior

Question

I'm attempting to add an overlay to an Android opengl ES application, and am currently specifying an ImageView and working off code from SurfaceViewOverlay API Demo, effectively merging two applications I wrote, one that was canvas based and one with opengl ES.

Due to merging two projects there was a large amount of copy paste involved and which lead to me accidentally using a null surface holder. Surprisingly this worked but threw a large number of warnings. Changing it to the surface holder of the opengl panel however causes the application to hang and not display either content from opengl or ImageView.

I played around with the code to make sure I wasn't initializing anything else where and confirmed the behavior on both a HTC Desire and Samsung Galaxy (both running 2.2)

Canvas c = null;
SurfaceHolder surfaceHolder2D = null; // this works but throws occational Null Pointer exceptions on the first canvas.drawText
//SurfaceHolder surfaceHolder2D = rsurfaceHolder; // this locks the application. 
//rsurfaceHolder is initialized with getHolder() in the opengl class and this worked with both when they where separate. 
//The change to ImageView was to get around issues with multiple SurfaceViews in an application having indeterminate Z order

try
{
    if (Global.RUNNING == 1)
    {
        c = surfaceHolder2D.lockCanvas(null); // when canvas is null this line can be ommited and it still works
        synchronized (surfaceHolder2D)
        {
            rpanel2D.onDraw(c);
        }
    }
    else
        sleep(1000);
}
catch (Exception e)
{
    e.printStackTrace();
}
finally
{
    if (c != null)
    {
        surfaceHolder2D.unlockCanvasAndPost(c);
    }
}

So what on earth is going on and what would be the correct way of performing this? I'm assuming that when I'm writing to a null canvas the compiler is doing something and sending it to the correct location regardless.

Leaving it as is, is not really an option, for several reason including not trusting that this will work across multiple versions of android, it flooding the logs with a constant stream of warnings with a significant frame rate drop and it only works if onDraw is invalidated on each call.

As for the onDraw function this simply is using canvas.drawText with a this.invalidate()

Curiously creating a new canvas and drawing to that and trying to invalidate or using setImageResource (either with a new or null) throws a "CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views" but not when drawing to a null canvas. In response I moved this to be called from the openGL thread however although this works for a few seconds, soon after the ImageView stops updating. It's on draw is still being called since a counter I placed in there to confirm it is incremented even though the display isn't. Hiding and unhiding the overlay results in it working again for a few seconds.

Any help would be greatly appreciated. If I'm on the wrong track completely please let me know. My alternative is to render this to BMP, create a new texture and draw it to a quad, but I imagine that would cause issues with thousands of textures created a minute.

Thanks in advance. -K

score 1 · Accepted Answer · answered Oct 10 '11 at 23:37

1

I am not sure what you are trying to achieve - wouldn't it be better if both renderings were done using OpenGL? Assuming that there is no other way, I would advice to create a FrameLayout and add the two views to it (GLSurfaceView and ImageView). If you need to change UI elements you have to do it from the UI Thread, changing elements ftom the GL Thread would cause an error; so be sure and do any change inside runOnUiThread Runnable.

answered Oct 10 '11 at 23:37

MichalisB

568
3
14

Hi MichalisB, this was pretty much the way I ended up going but I might return to pure opengl. I was trying to build a dynamic ui heavy opengl app that needed to run fast, on both low end and high end devices. The HTC Desire experiences quite a frame rate drop pushing too many quads so overlaying a canvas that I would only have to update every 1/5th of a second seemed like a good idea at the time. At present it updates every frame as part of the android looper which cuts performance in half but I'll see what I can do with it later on. Thanks – Kactus Oct 13 '11 at 02:11
I have no idea how you do you 2D rendering, so this might be redudant if you have already done this, but have you tried optimizing it? For example, dont change textures by using a texture atlas and/or draw the whole UI with just one draw call by using degenerate triangle strips? – MichalisB Oct 14 '11 at 01:30
My main issue is that not only does the interface change, but the labels them selves display numerical data that is constantly changing. I had created a numerical tex array/display wrapper class so I could easily display float values to x decimal places etc. Was faster than the alts I benchmarked originally but as requirements have grown that now means Several hundred tex bind and quad draw operations each frame for that alone. My alternative thinking is that I could have a single Bitmap and use canvas tools. It seems to scale better for a lot of dynamic text but initial overheads are steep. – Kactus Oct 14 '11 at 02:42

Android ImageView overlay over Opengl ES unexpected behavior

1 Answers1