Why are distances from centers of the same edges different and produce "T junctions" while tessellating an icosahedron with OpenGL?

Question

I am trying to tessellate an already subdivided icosahedron using OpenGL tessellation shaders. I want to tessellate triangles closer to camera more and avoid T junctions as much as possible.

I tried calculating centers of every edge and center of the original triangles. Everything in view space. I believe Z component of every position I calculated should be the distance and the distance should be the same on different triangles when it's the same edge. However my solution is producing T junctions on the edges of the original triangles, which are especially visible if I don't use fractional spacing.

This is what I am getting for equal_spacing Equal Spacing Tessellation

For fractional_odd_spacing the T junctions are smaller, but still are a problem when trying to apply height map

Without Height Map

With Height Map Applied

I don't know if it is because:

• the distances are calculated correctly, are somewhat the same, but are used to calculate tessellation level for a different edge (wrong index in gl_TessLevelOuter);
• I am missing something and/or not understanding some concept of OpenGL tessellation;
• float data type limitations when calculating the positions;
• something else that I haven't thought of.

If my solution looks correct, how can I even try to debug something like that?

Beneath are versions of my shader programs that include only parts that define final vertex positions.

Vertex shader

#version 430 core

layout (location = 0) in vec3 inPos;

uniform mat4 _modelMat;
uniform mat4 _viewMat;

out vec3 csPos;
out vec3 csVPos;

void main() {
    csPos = inPos;
    csVPos = vec3(_viewMat * _modelMat * vec4(inPos, 1.0));
}

Tessellation Control Shader

#version 430 core

layout (vertices = 3) out;

in vec3 csPos[];
in vec3 csVPos[];

out vec3 esPos[];

uniform mat4 _modelMat;
uniform mat4 _viewMat;

void main() {
    esPos[gl_InvocationID] = csPos[gl_InvocationID];

    if(gl_InvocationID == 0) {
        float camDist;
        // variable for me to control how fast it's tessellating when going closer
        // to the triangles
        const float tessFactor = 1.0;

        // calculating outer tessellation levels
        for(int vertexInd = 0; vertexInd < 3; vertexInd += 1) {
            vec3 edgePos = 0.5 * (csVPos[vertexInd] + csVPos[(vertexInd+1)%3]);
            camDist = -edgePos.z;

            gl_TessLevelOuter[vertexInd] = tessFactor / camDist;
        }

        // calculating inner tessellation level
        vec3 triangleCenter = 0.33333 * (csVPos[0] + csVPos[1] + csVPos[2]);
        camDist = -triangleCenter.z;
        gl_TessLevelInner[0] = tessFactor / camDist;
    }
}

Tessellation Evaluation Shader

#version 430 core

layout (triangles, fractional_odd_spacing, ccw) in;

in vec3 esPos[];

void main() {
    vec3 vertexPos = vec3(0.0);
    vertexPos += esPos[0] * gl_TessCoord.x;
    vertexPos += esPos[1] * gl_TessCoord.y;
    vertexPos += esPos[2] * gl_TessCoord.z;
    vertexPos = normalize(vertexPos);

    gl_Position = vec4(vertexPos, 1.0);
}

Geometry Shader

#version 430 core

layout (triangles) in;
layout (triangle_strip, max_vertices = 3) out;

uniform mat4 _modelMat;
uniform mat4 _viewMat;
uniform mat4 _projMat;

void main() {
    for(int vertexInd = 0; vertexInd < 3; vertexInd += 1) {
        gl_Position = _projMat * _viewMat * _modelMat * gl_in[vertexInd].gl_Position;
        EmitVertex();
    }

    EndPrimitive();
}

Answer 1

    vertexPos += esPos[0] * gl_TessCoord.x;
    vertexPos += esPos[1] * gl_TessCoord.y;
    vertexPos += esPos[2] * gl_TessCoord.z;

The three outer tessellation levels (0, 1, 2) specify the tessellation of the three edges of the abstract triangle. However, you must pay attention to how these edges map to the actual barycentric coordinates of this triangle.

Your code generates tessellation level 0 based on the edge defined by the vertex indices 0 and 1. And your TES code to generate the vertex position multiplies vertex index 0 by the X component of the barycentric coordinate. But... in the abstract triangle, the X component of the barycentric coordinate represents the vertex opposite of the edge tessellated by tessellation level 0. This means that, along the edge defined by level 0, only vertices 1 and 2 will have a contribution.

You applied the tessellation to the wrong edges. Or said another way, you applied the wrong vertices to the tessellated patch.

So let's break down the diagram of the abstract triangle mapping. Each tessellation level specifies an edge, so "edge 0" means the edge for tessellation level 0. From the diagram, we can see that the edges 2 and 0 meet at the vertex defined by the Y component of the barycentric coordinate. Edges 0 and 1 meet at the Z component of the coordinate. And edges 1 and 2 meet at the X component.

You generate the tessellation using the following mapping:

Edge	vertices
0	0, 1
1	1, 2
2	2, 0

But given this, your vertex 0 represents the barycentric coordinate's Y component. Vertex 1 is maps to Z component. And vertex 2 maps to the barycentric coordinate's X component.