How to use stenciling to create a 2d clipping mask in webgl?

Question

I've been investigating a way to create a clipping mask similar to this one (done in SVG).

Based on my findings I chose to achieve this through stenciling. My implementation however is grossly incorrect. I'm not entirely sure how gl.stencilOp and gl.stencilFunc work because it seems like I need to render the fragment masking my main content twice. Once before I render the main content and once after with varying parameters.

Here's the working test: https://dl.dropboxusercontent.com/u/1595444/experiments/two.js/issues/issue-56/stencil-buffer/test/clip.html

enter image description here

The relevant snippet / portion of this test can be found in ../src/renderers/webgl.js starting at L67:

if (this._mask) {

  gl.enable(gl.STENCIL_TEST);
  gl.stencilFunc(gl.ALWAYS, 1, 1);

  gl.colorMask(false, false, false, true);
  gl.stencilOp(gl.KEEP, gl.KEEP, gl.INCR);

  // Renders the mask through gl.drawArrays L111
  webgl[this._mask._renderer.type].render.call(
    this._mask, gl, program, this);

  gl.colorMask(true, true, true, true);
  gl.stencilFunc(gl.NOTEQUAL, 0, 1);
  gl.stencilOp(gl.KEEP, gl.KEEP, gl.KEEP);

}

// Renders main content through a series of gl.drawArrays calls
_.each(this.children, webgl.group.renderChild, {
  gl: gl,
  program: program
});

if (this._mask) {

  gl.colorMask(false, false, false, false);
  gl.stencilOp(gl.KEEP, gl.KEEP, gl.DECR);

  // Re-render mask so main content doesn't flicker
  webgl[this._mask._renderer.type].render.call(
    this._mask, gl, program, this);

  gl.colorMask(true, true, true, true);
  gl.stencilFunc(gl.NOTEQUAL, 0, 1);
  gl.stencilOp(gl.KEEP, gl.KEEP, gl.KEEP);

  gl.disable(gl.STENCIL_TEST);

}

Guidance to emulate the webgl stenciling to work like the svg example would be much appreciated.

score 3 · Answer 1 · answered Jul 23 '14 at 08:50

What you need to do is:

Draw your stencil area (in your case the blue rectangle),
Stop drawing into the stencil
Draw your scene which you want to consider the stencil
Stop stencil

As follow:

if (this._mask) {
    // Clearing the stencil buffer
    gl.clearStencil(0);
    gl.clear(gl.STENCIL_BUFFER_BIT);

    // Replacing the values at the stencil buffer to 1 on every pixel we draw
    gl.stencilFunc(gl.ALWAYS, 1, 1);
    gl.stencilOp(gl.REPLACE, gl.REPLACE, gl.REPLACE);

    // disable color (u can also disable here the depth buffers)
    gl.colorMask(false, false, false, false);

    gl.enable(gl.STENCIL_TEST);

    // Renders the mask through gl.drawArrays L111
    webgl[this._mask._renderer.type].render.call(this._mask, gl, program, this);

    // Telling the stencil now to draw/keep only pixels that equals 1 - which we set earlier
    gl.stencilFunc(gl.EQUAL, 1, 1);
    gl.stencilOp(gl.KEEP, gl.KEEP, gl.KEEP);
    // enabling back the color buffer
    gl.colorMask(true, true, true, true);
}

// Renders main content through a series of gl.drawArrays calls
_.each(this.children, webgl.group.renderChild, {
   gl: gl,
   program: program
});

// Stop considering the stencil
if (this._mask) {
   gl.disable(gl.STENCIL_TEST);
}

Sorry for the delay and thanks for the help! This is much closer, but still not quite correct. The circle and the rectangle are both textures on top of quads. You're implementation masks the quad, but not the texture (the circle). Here's an updated version with your implementation: https://dl.dropboxusercontent.com/u/1595444/experiments/Two.js/issues/issue-56/raziza/index.html — jonobr1, Sep 09 '14 at 20:21
@jonobr1 That link is no longer valid. Please embed the code in an answer post. — ssokolow, Sep 04 '18 at 19:28

How to use stenciling to create a 2d clipping mask in webgl?

1 Answers1