0

I'm programming a DX11 SM5.0 terrain hull shader along the lines of many examples such as Frank Luna's. While investigating the reason for crazy flickering of the (wireframe) terrain, I've homed in on what seems to be a problem with the pow() function.

My code to calculate tessellation factors is:

float CalcTessFactor(float3 p)
{
    float d = distance(p, cameraPosition);
    float s = saturate((d - 1000.0f) / (5000.0f - 1000.0f));
    //return pow(2, (lerp(6, 1, s)));
    return saturate((5000.0f - d) / 5000.0f)*64.0f;
}

The hard-coded numeric constants are part of my debugging reduction. The commented-out line: "//return pow(..." is the original code, which I've replaced by the line that now follows.

With this replacement the tessellation is completely stable and reduces with distance from the camera. As expected, the reduction is only linear rather than logarithmic, but at least it works, is nicely tessellated and shows no signs of flicker.

With the original code, the mesh appeared to be switching at frame rate between apparently random tessellation factors.

Can anyone suggest what might be going wrong?

My patch constant function is:

struct HullInputType
{
    float3 position : POSITION;
    float4 color : COLOR;
};

struct ConstantOutputType
{
    float edges[4] : SV_TessFactor;
    float inside[2] : SV_InsideTessFactor;
};

ConstantOutputType TerrainPatchConstantFunction(InputPatch<HullInputType, 4> patch, uint patchId : SV_PrimitiveID)
{
    ConstantOutputType output;

    // Compute midpoint on edges, and patch center
    // order of vertices is: 0 1
    //                       2 3
    float3 e0 = 0.5f*(patch[0].position + patch[2].position);
    float3 e1 = 0.5f*(patch[0].position + patch[1].position);
    float3 e2 = 0.5f*(patch[1].position + patch[3].position);
    float3 e3 = 0.5f*(patch[2].position + patch[3].position);
    float3 c = 0.25f*(patch[0].position + patch[1].position + patch[2].position + patch[3].position);

    // Set the tessellation factors for the four edges of the quad.
    output.edges[0] = CalcTessFactor(e0);
    output.edges[1] = CalcTessFactor(e1);
    output.edges[2] = CalcTessFactor(e2);
    output.edges[3] = CalcTessFactor(e3);

    // Set the tessellation factor for tessallating inside the quad.
    output.inside[0] = CalcTessFactor(c);
    output.inside[1] = output.inside[0];

    return output;
}
Barbarian
  • 237
  • 4
  • 9

1 Answers1

1

Looking at your return value. Looks like you are doing your tesselation backwards. Uncomment the price of code. Hard code the distance value say to say 1000. You should get a consistent tesselation. If you do then you should get no flickering. If you don't then its something with the power function as you assert or out not then your distance vectors are funky.

Edit: Added my tessellation function for reference.

  struct VertexOut
{
    float4 PosW : POSITION0;
    float4 waterAttributes : POSITION2;
    float4 direction : POSITION1;
//    float4 wind : POSITION2;
    float tessFactor : TESS;
};

//========================================================================================================================

[domain("tri")]
[partitioning("integer")]
[outputtopology("triangle_cw")]
[outputcontrolpoints(3)]
[patchconstantfunc("PatchHS")]
HullOut HSMain(InputPatch<VertexOut, 3> p,
           uint i : SV_OutputControlPointID,
           uint patchId : SV_PrimitiveID)
{
    HullOut hout;

    // Pass through shader.
    hout.PosW = p[i].PosW;
    hout.direction = p[i].direction;
    hout.waterAttributes = p[i].waterAttributes;
 //   hout.wind = p[i].wind;

    return hout;
}

PatchTess PatchHS(InputPatch<VertexOut, 3> patch,
                  uint patchID : SV_PrimitiveID)
{
    PatchTess pt;

    // Average tess factors along edges, and pick an edge tess factor for 
    // the interior tessellation.  It is important to do the tess factor
    // calculation based on the edge properties so that edges shared by 
    // more than one triangle will have the same tessellation factor.  
    // Otherwise, gaps can appear.
    pt.EdgeTess[0] = 0.5f * (patch[1].tessFactor + patch[2].tessFactor);
    pt.EdgeTess[1] = 0.5f * (patch[2].tessFactor + patch[0].tessFactor);
    pt.EdgeTess[2] = 0.5f * (patch[0].tessFactor + patch[1].tessFactor);
    pt.InsideTess = pt.EdgeTess[0];

    return pt;
}
ErnieDingo
  • 444
  • 5
  • 11
  • I want the greatest tessellation close to the camera. Anyway I tried your suggestion and yes, hard coded distance values produce consistent results. I was going to post my patch constant function, but a comment doesn't seem to accept code, so I've edited my original question to add it. – Barbarian Mar 21 '18 at 21:16
  • certainly interesting. If setting provides consistent results, could it be your distance vectors that are at issue? My tesselation function isn't as complicated as yours. Usually I don't have each edge tessellated individually. Will check my tessellation function. I use Frank Luna's tutorials also. – ErnieDingo Mar 21 '18 at 21:32
  • Edit: Added in my code to the answer, as I thought, I don't calculate my tessellation in the Hullshader, I do it at the vertex shader. hence the difference.You're also doing quads, so fundamentally different – ErnieDingo Mar 21 '18 at 21:38
  • 1
    Thanks, Ernie. I hadn't thought about doing the tess calcs in the vertex shader. It's probably slightly more efficient too, and since shared edges also share both vertices the results should be consistent between quads. I'll give it a go. – Barbarian Mar 21 '18 at 22:55
  • 1
    Well, I still don't know what was wrong with the original hull shader method, but I've migrated the CalcTessFactor function to the vertex shader and it works like a charm. The VS calls it once per vertex, where the hull shader called it five times per quad. So that's a double win. I'll accept your answer and thanks for your help. – Barbarian Mar 21 '18 at 23:11
  • On reflection i should of realised that your distance calculation would vary per edge In the hull shader. By moving to vertex shader will always reflect a consistent result. Fault on myself for not thinking more deeply about what was going on in there. – ErnieDingo Mar 22 '18 at 03:23