I am trying to create a billiards simulation and have been using glReadPixels along with gluUnProject to project my mouse pointer into the scene.
This works fine if the mouse is pointing at an object in the scene (the table for instance) but when it points to the background it messes up the gluUnProject due to the glReadPixels call returning 1.0.
I'm trying to figure out how to draw a transparent plane at the same level of the table so that no matter where I point the mouse in my scene, it will get the depth as if it were pointing onto the same plane as the table.
If I draw a transparent quad without glAlphaFunc(GL_GREATER, 0.01f); it will draw the quad as white and the depth testing will work as I planned, but when I add in the call to alphaFunc to get the quad to be transparent, the depth goes back to what it was before. From what I've seen, glReadPixels reads the pixels from the frame buffer, so this makes sense, I'm just wondering how I can work around this.
I've also tried reversing the winding on the quad so that it wouldn't be visible from above, but this has the same problem of glReadPixels taking it's measurements from the framebuffer.
In short, how do I get glReadPixels to get it's depth component from an object without drawing that object to the screen?
Here's the calls to glReadPixels and gluUnProject:
winX = (float)x;
winY = (float)viewport[3] - (float)y;
glReadPixels(x, (int) winY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ);
gluUnProject(winX, winY, winZ, modelview, projection, viewport, &posX, &posY, &posZ);