Stop two shadows overlapping using the stencil buffer in OpenGL

Question

I have two planar shadows of the same object coming from the same light source - one that casts on the floor and one to cast on the wall when the object is close enough. Everything works just fine as far as the shadows being cast, I'm using the stencil buffer to make sure that the two shadows only cast on their respective surfaces without being rendered outside of the room.

The problem is that the two stencil buffers bleed into each other, specifically whichever shadow I render second bleeds into the stencil buffer for the first one. I figure it's some issue with the stencil function or something, using the wrong parameters, but I can't seem to figure it out.

// Generate the shadow using a shadow matrix (created using light position and vertices of 
// the quad on which the shadow will be projected) and the object I'm making a shadow of

void createShadow(float shadowMat[16])
{
    glDisable(GL_DEPTH_TEST);
    glDisable(GL_LIGHTING);
    glDisable(GL_TEXTURE_2D);

    // Set the shadow color
    glColor3f(0.1, 0.1, 0.1);

    glPushMatrix();
    
    // Create the shadow using the matrix and the object casting a shadow
    glMultMatrixf((GLfloat*)shadowMat);

    translate, rotate etc;
    render object;

    glPopMatrix();

    // Reset values to render the rest of the scene
    glColor3f(1.0, 1.0, 1.0);
    glEnable(GL_DEPTH_TEST);
    glEnable(GL_LIGHTING);
    glEnable(GL_TEXTURE_2D);

}

// Set up the stencil buffer and render the shadow to it
void renderShadow(float shadowMat[16], float shadowQuad[12])
{
    glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
    glEnable(GL_STENCIL_TEST);
    glStencilFunc(GL_ALWAYS, 1, 1);
    glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
    glDisable(GL_DEPTH_TEST);

    // Create a stencil for the shadow, using the vertices of the plane on which it will 
    // be projected
    glPushMatrix();
    translate, rotate etc;

    glEnableClientState(GL_VERTEX_ARRAY);
    // The shadow quad is the same vertices that I use to make the shadow   matrix
    glVertexPointer(3, GL_FLOAT, 0, shadowQuad);

    glDrawArrays(GL_QUADS, 0, 4);

    glDisableClientState(GL_VERTEX_ARRAY);

    glPopMatrix();

    glEnable(GL_DEPTH_TEST);
    glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
    glStencilFunc(GL_EQUAL, 1, 1);
    glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

    // Render the shadow to the plane
    createShadow(shadowMat);

    glDisable(GL_STENCIL_TEST);
}

// In the render function:
Render floor/surrounding area;
Set up light using the same position used to make the shadow matrix;

renderShadow(wallShadowMatrix, wallVertices);
renderShadow(floorShadowMatrix, floorVertices);

Render rest of scene;

If I render the shadows on their own they work as intended, but when I render them together, whichever one rendered second shows up in the stencil of the first shadow.

I've included a few pictures; the first two show the individual Shadow on the wall and Shadow on the floor, and here is the floor shadow rendered after the wall shadow, and vice versa.

Answer 1

Fixed it, I needed to add the following code between the two renderShadow calls in the render function:

glClear(GL_STENCIL_BUFFER_BIT);