0

enter image description here

I want to mask a rectangle shape with two different shapes

Shape 1 Which is in shape of a circle

Shape 2 Which is in shape of rectangle

This is the code for setting stencil mask for shape 1

glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP);
glStencilFunc(GL_NEVER, 1, 1); // never pass stencil test
glStencilMask(1);

This is the code for setting stencil mask for shape 2

glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP);
glStencilFunc(GL_NEVER, 4, 4); // never pass stencil test
glStencilMask(4);

Now i can mask the geometry shape with a value of 5 and the shape will only be drawn where both the masks were drawn

 glStencilFunc(GL_EQUAL, 5, 5);
 break;

But how do i draw the geometry where Shape 1 is GL_EQUAL and Shape 2 is GL_NOTEQUAL

Like the image below

enter image description here

Summit
  • 2,112
  • 2
  • 12
  • 36

1 Answers1

2

Instead of asking where "Shape 2 is GL_NOTEQUAL", ask what value it will be EQUAL there. If you're interested in pixels that are in Shape 1 and not in Shape 2, then the value of the stencil buffer there will be 1; that is 1 for Shape 1 plus 0 for Shape 2. You can do that test by modifying the ref argument to glStencilFunc:

glStencilFunc(GL_EQUAL, 1, 5);
Yakov Galka
  • 70,775
  • 16
  • 139
  • 220
  • i have tried tried this but it does not work the way i want it to work , the circle should be drawn where we have mask 1 but if mask 2 two also happens to be overlapping on the same area as mask 1 than again the object should not be drawn in that region. – Summit Feb 02 '22 at 02:42
  • 1
    @Summit: yes, that's exactly what it's doing. perhaps you should post the code that you tried but didn't work. – Yakov Galka Feb 02 '22 at 03:25