Calculating Per face normals in Geometry Shader

Question

I'm trying to calculate per-face normals in geometry shader, using the following pipeline

//VERTEX_SHADER
#version 330
layout(location = 0) in vec4 vertex;
out vec3 vert;
uniform mat4 projMatrix;
uniform mat4 mvMatrix;

void main()
{
  vert = vertex.xyz;
  gl_Position = projMatrix * mvMatrix * vertex;
}

//GEOMETRY_SHADER
#version 330

layout ( triangles ) in;
layout ( triangle_strip, max_vertices = 3 ) out;

out vec3 normal_out;
uniform mat3 normalMatrix;

void main()
{
    vec3 A = gl_in[2].gl_Position.xyz - gl_in[0].gl_Position.xyz;
    vec3 B = gl_in[1].gl_Position.xyz - gl_in[0].gl_Position.xyz;
    normal_out = normalMatrix * normalize(cross(A,B));
    gl_Position = gl_in[0].gl_Position;
    EmitVertex();
    gl_Position = gl_in[1].gl_Position;
    EmitVertex();
    gl_Position = gl_in[2].gl_Position;
    EmitVertex();
    EndPrimitive();
}

//FRAG_SHADER
#version 330
in vec3 normal_out;
in vec3 vert;
out vec4 fColor;
uniform vec3 lightPos;

void main()
{
    highp vec3 L = normalize(lightPos - vert);
    highp float NL = max(dot(normal_out, L), 0.0);
    highp vec3 color = vec3(1, 1, 0.0);
    fColor = vec4(color*NL, 1.0);
}

However, I end up with very weird looking faces that keeps flickering(I included a snapshot below). It occurred to me that it might be because I'm using 8 vertices to represent 1 cell(cube) instead of 24 vertices, But I'm not quite sure if that is what is causing the problem.

Left: Using Light Weighting 'NL', Right:Without

Nicol Bolas · Answer 1 · 2016-01-14T19:47:53.357

2

After every call to EmitVertex, the contents of all output variables are made undefined. Therefore, if you want to output the same value to multiple vertices, you must copy it to the output every time.

Also, note that each shader stage's outputs provide inputs only to the next stage. So if you have a GS, and you want to pass a value from the VS to the FS, you must have the GS explicitly pass that value through.

edited Jan 14 '16 at 19:47

answered Jan 14 '16 at 18:05

Nicol Bolas

449,505
63
781
982

Thanks for your timely reply, would you please clarify more? given my case for example – Mostafa Wasat Jan 14 '16 at 18:17
Do you mean output variables from Vertex shader or the geometry shader ? – Mostafa Wasat Jan 14 '16 at 18:22
The geometry shader. Vertex shaders can't call `EmitVertex`, after all. And I'm not sure how much more I can clarify without actually writing it for you. – Nicol Bolas Jan 14 '16 at 19:03
I tried setting the 'normal_out' variable before emitting each vertex, The output is almost the same.. I know vertex shaders cant call EmitVertex but i have an output variable 'vert' that will be used in the fragment shader, will that also be undefined after each EmitVertex in the GS ? – Mostafa Wasat Jan 14 '16 at 19:20
1

@MostafaWasat: I don't know how your shader is even compiling and linking properly then. Because the only input variables that your fragment shader receives are the ones produced by the last active vertex processing stage. In your case, the geometry shader. So if your GS does not output `vert`, the FS won't get one. And that should have given you a linker error. – Nicol Bolas Jan 14 '16 at 19:46
Well idk, it complies and runs without any errors. I too had my doubts about outputting variables directly from VS to FS bypassing the GS. so THANK YOU for confirming that. Can you please tell me how I can pass these values to GS then to FS? I cant seem to be able to do it. – Mostafa Wasat Jan 14 '16 at 19:56
When i try to access 'vert' in GS using 'in vec3 vert[]' it gives me a compilation error – Mostafa Wasat Jan 14 '16 at 20:15

Calculating Per face normals in Geometry Shader

1 Answers1