I am trying to implement frustum culling in my OpenGL 2d game. The only kind of geometric objects in my game at this point are rectangles, so I thought this would be pretty easy, but I am getting unexpected results. I have setup a symmetrical perspective projection with a field-of-view angle of 45 degrees, and near and far planes at 0.01 and 50 respectively. The eye vector is always parallel to the z-axis, the camera can only move along the x and y axes.
My idea was to get the rectangular area of the world space that is currently visible to the camera at the z-coordinate of the rectangle I am trying to cull. Because the camera is looking at the center of the frustum, I calculate the distance to the edges of this visible rectangular area as follows:
GLfloat maxDistance = givenRectangle.z * tanf(0.5 * (fovAngle * M_PI/180) );
Then I add and substract this distance to and from the x and y coordinates of the camera to get the maximum and minimum visible x and y, and then test the given rectangle to see if it's in between these values.
My question is whether I am on the right track here, and why the above formula returns an absurdly small value (something*10^-37) when I have an object at z=5, which should clearly be visible with the Camera at (0,0,0)?