How do I update generate a normal map to match an equirectangular projection dilacement map

Question

Im rendering a geodesic sphere of radius 1 in OpenGL and in my glsl tesselation shader multiplying the vertex by a value from a height/ displacement map which creates some large rays protruding from the sphere. I am looking for a way to be able to clearly see the geometry this creates. Some diffuse lighting or outlining or something like a sobel filter would be ideal, but I am having trouble calculating the normals needed to apply the lighting or outlining.

I was able to calculate the surface normals in a geometry shader, but this created an un acceptable bottleneck as even with it mostly just being a passthrough shader the large number of polygons bogged the gs down. I am stuck with OpenGL 4.2 so nVidias passthrough extension is unavailable to me.

I was thinking maybe a pre computed normal map might be the way to go, but I am not sure how to generate that based on my equirectangular projection displacement map. Any thoughts or helpful advice?

Answer 1

I was able to find this Sobel Filter for creating a normal map from a displacement map. It isn't perfect as I still have to slew normals around the sphere, but it is pretty good.

vec4 GenerateNormal(int imgWid, int imgHei, GLuint *displacementMap, int texX, int texY)
{
    float normalStrength = 8;
    int xCoord = texX;// (int)(texX*(float)imgWid);
    int yCoord = texY;// (int)(texY*(float)imgHei);
    float tl = abs(GetDisplacement(displacementMap, texX, texY, imgWid, imgHei, -1, -1));   // top left
    float  l = abs(GetDisplacement(displacementMap, texX, texY, imgWid, imgHei, -1, 0));   // left
    float bl = abs(GetDisplacement(displacementMap, texX, texY, imgWid, imgHei, -1, 1));   // bottom left
    float  t = abs(GetDisplacement(displacementMap, texX, texY, imgWid, imgHei, 0, -1));   // top
    float  b = abs(GetDisplacement(displacementMap, texX, texY, imgWid, imgHei, 0, 1));   // bottom
    float tr = abs(GetDisplacement(displacementMap, texX, texY, imgWid, imgHei, 1, -1));   // top right
    float  r = abs(GetDisplacement(displacementMap, texX, texY, imgWid, imgHei, 1, 0));   // right
    float br = abs(GetDisplacement(displacementMap, texX, texY, imgWid, imgHei, 1, 1));   // bottom right
    // Compute dx using Sobel:
    //           -1 0 1 
    //           -2 0 2
    //           -1 0 1
    float dX = tr + 2 * r + br - tl - 2 * l - bl;
    // Compute dy using Sobel:
    //           -1 -2 -1 
    //            0  0  0
    //            1  2  1
    float dY = bl + 2 * b + br - tl - 2 * t - tr;
    // Build the normalized normal
    vec4 N = vec4(normalize(vec3(dX, 1.0f / normalStrength, dY)), 1.0f);
    //convert (-1.0 , 1.0) to (0.0 , 1.0), if needed
    return normalize(N * 0.5f + 0.5f);
}