3

This is a continuation of my previous problem and post, seen here. Thanks to the answer I received there I feel I was able to get a little closer to my goal, as well as further my learning of OpenGL, but shortly after figuring out the basics of working with stencil buffers, I've run into a problem.

It seems that when I draw a sprite to the stencil buffer, it draws the entire square area, rather than just the pixels that aren't fully transparent as I had ignorantly hoped. I vaguely understand why it happens that way, but I am not sure where the solution lies. I have experimented with the stencil itself quite a bit, and I have modified the shaders that the spritebatch uses to discard low-alpha fragments, but I seem to be failing to see the bigger picture.

As a visual example of the problem, I will continue with the examples I used in the previous question. Right now, trying to draw two circles over each other (So they blend perfectly, no overlapping), I am getting this :

Stencil testing

So, basically, is there a way for me to utilize stencil buffers using the Sprite and SpriteBatch functionality of LibGDX on complex shapes (Circles are only being used as an example), or do I need to look for an alternative route?

EDIT ::

    Gdx.gl.glClearColor(1, 1, 1, 1);
    Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT | GL20.GL_DEPTH_BUFFER_BIT);

    Gdx.gl.glEnable(GL20.GL_BLEND);
    Gdx.gl.glBlendFunc(GL20.GL_SRC_ALPHA, GL20.GL_ONE_MINUS_SRC_ALPHA);
    Gdx.gl.glEnable(GL20.GL_DEPTH_TEST);
    Gdx.gl.glDepthMask(true);

    batch.begin();
    sprite.draw(batch);
    sprite2.draw(batch);
    batch.end();

    Gdx.gl.glDisable(GL20.GL_DEPTH_TEST);
Community
  • 1
  • 1
Synthetik
  • 219
  • 3
  • 11

2 Answers2

1

Stencil writing/testing happens at the fragment level. When you draw a circle, you are actually drawing a quad of fragments where each fragment may or may not be textured. The issue of the matter is that the GPU doesn't care what color you write into the fragment when it does the stencil test for the fragment. Therefore, you need to discard fragments to stop the stencil writes for the parts of the quad where you don't want to write stencil values.

TL;DR Use "if(gl_FragColor.a < 0.01) discard;" in the fragment shader to make sure a circle of fragments (and stencil values) are generated.

Darth Zaloj
  • 153
  • 6
0

Mby this helps. I solve same problem with blending and depth test. First you need clear depth and color bit.

Gdx.gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);

Then you need enable some gl features:

Gdx.gl.glEnable(GL20.GL_BLEND);
Gdx.gl.glBlendFunc(GL20.GL_SRC_ALPHA, GL20.GL_ONE_MINUS_SRC_ALPHA);
Gdx.gl.glEnable(GL10.GL_DEPTH_TEST);
Gdx.gl.glDepthMask(true);

After that you can draw your objects. In my project smtng like this:

shapeRenderer.setColor(c.getR(), c.getG(), c.getB(), FIELD_ALPHA);
shapeRenderer.filledCircle(p.getPos().x + s.getOffset().x, ApplicationEnv.SCREEN_HEIGHT - p.getPos().y + s.getOffset().x, b.getRadius());

Important! After drawing you call end() for your spriteBatch (or something else) and disable depth_test feature:

Gdx.gl.glDisable(GL10.GL_DEPTH_TEST);

Result: Before After

user2329816
  • 1
  • Your result is pretty much right on with what I'm after, but there seems to be something missing from the example. Using the above code didn't seem to change the display at all from what you normally get when layering transparent sprites (Similar to your "before" image). I've tinkered with depth testing in different ways as well, trying to solve this, but I can't seem to find out what I am missing to make it work, even with a simple setup that does nothing but layer two sprites. At the very least, thank you for showing me that the effect can be achieved this way. – Synthetik May 06 '13 at 23:19
  • Maybe you don't set this Gdx.gl.glClearColor(0.9f, 0.9f, 0.9f, 1); - fill background with gray color in my case. I use depth test only when i draw circles without "overlapping". Squares and other objects drawed by SpriteBatch without DEPTH_TEST. Or you can send your piece of code here or to me. – user2329816 May 10 '13 at 21:01
  • Sorry about the delays, been so darn busy lately! At the moment I am using a bare bones test project to try and figure this out, above I posted what amounts to my entire render method when I do what you posted earlier, but the result is simply two overlapping circles, similar to your "before" image. Basically, simply enabling depth testing like that doesn't seem to be changing how it's drawing at all, so I think I am missing some key setup line in between there, but as much as I have been experimenting with it, I cannot find what I am missing. – Synthetik May 13 '13 at 18:00