I'm programming a DX11 SM5.0 terrain hull shader along the lines of many examples such as Frank Luna's. While investigating the reason for crazy flickering of the (wireframe) terrain, I've homed in on what seems to be a problem with the pow() function.
My code to calculate tessellation factors is:
float CalcTessFactor(float3 p)
{
float d = distance(p, cameraPosition);
float s = saturate((d - 1000.0f) / (5000.0f - 1000.0f));
//return pow(2, (lerp(6, 1, s)));
return saturate((5000.0f - d) / 5000.0f)*64.0f;
}
The hard-coded numeric constants are part of my debugging reduction. The commented-out line: "//return pow(..." is the original code, which I've replaced by the line that now follows.
With this replacement the tessellation is completely stable and reduces with distance from the camera. As expected, the reduction is only linear rather than logarithmic, but at least it works, is nicely tessellated and shows no signs of flicker.
With the original code, the mesh appeared to be switching at frame rate between apparently random tessellation factors.
Can anyone suggest what might be going wrong?
My patch constant function is:
struct HullInputType
{
float3 position : POSITION;
float4 color : COLOR;
};
struct ConstantOutputType
{
float edges[4] : SV_TessFactor;
float inside[2] : SV_InsideTessFactor;
};
ConstantOutputType TerrainPatchConstantFunction(InputPatch<HullInputType, 4> patch, uint patchId : SV_PrimitiveID)
{
ConstantOutputType output;
// Compute midpoint on edges, and patch center
// order of vertices is: 0 1
// 2 3
float3 e0 = 0.5f*(patch[0].position + patch[2].position);
float3 e1 = 0.5f*(patch[0].position + patch[1].position);
float3 e2 = 0.5f*(patch[1].position + patch[3].position);
float3 e3 = 0.5f*(patch[2].position + patch[3].position);
float3 c = 0.25f*(patch[0].position + patch[1].position + patch[2].position + patch[3].position);
// Set the tessellation factors for the four edges of the quad.
output.edges[0] = CalcTessFactor(e0);
output.edges[1] = CalcTessFactor(e1);
output.edges[2] = CalcTessFactor(e2);
output.edges[3] = CalcTessFactor(e3);
// Set the tessellation factor for tessallating inside the quad.
output.inside[0] = CalcTessFactor(c);
output.inside[1] = output.inside[0];
return output;
}