OpenGL layered rendering interferes with layer 0

Question

I am using gl_Layer = gl_InvocationID; in a geometry shader to render into a framebuffer with a 3D texture attached.

This mostly works fine. Except every invocation of the shader also renders into layer 0, as well as the layer that I specify.

How can I avoid this? Is there something vital I'm missing with setting up the framebuffer? Perhaps with glFramebufferTexture?

Geometry Shader

#version 400

layout(invocations = 32) in;
layout(points) in;
layout(triangle_strip, max_vertices = 3) out;

out vec3 raster_color;

float blue;

void main()
{
  gl_Layer = gl_InvocationID;
  blue = float(gl_InvocationID) / 31.0;

  gl_Position  = vec4( -1.0, -1.0, 0.0, 1.0 );
  raster_color = vec3( 0.0, 0.0, blue );
  EmitVertex();

  gl_Position  = vec4( 1.0, -1.0, 0.0, 1.0 );
  raster_color = vec3( 1.0, 0.0, blue );
  EmitVertex();

  gl_Position  = vec4( 0.0, 1.0, 0.0, 1.0 );
  raster_color = vec3( 1.0, 1.0, blue );
  EmitVertex();

  EndPrimitive();
}

Fragment Shader

#version 400

in vec3 raster_color;
out vec4 fragment_color;

void main()
{
  fragment_color = vec4( raster_color, 1.0 );
}

Answer 1

EmitVertex invalidates all per-vertex outputs after it returns. The most obvious per-vertex outputs in this shader are:

1. raster_color
2. gl_Position

But, you may not have realized that gl_Layer is also per-vertex or which vertex this needs to be set for.

---

gl_Layer will be undefined for every vertex after the first in this shader. Some drivers will leave it untouched and simply work, others will do anything they want with it and you cannot make any assumptions about gl_Layer after EmitVertex (...). You are playing with fire, because it may not be the first vertex that defines a primitive's layer (more on this later).

To fix this, re-write your geometry shader this way:

#version 400

layout(invocations = 32) in;
layout(points) in;
layout(triangle_strip, max_vertices = 3) out;

out vec3 raster_color;

float blue;

void main()
{
  blue = float(gl_InvocationID) / 31.0;

  gl_Position  = vec4( -1.0, -1.0, 0.0, 1.0 );
  raster_color = vec3( 0.0, 0.0, blue );
  gl_Layer = gl_InvocationID; // Handle case where First Vertex is Layer Provoking
  EmitVertex();

  gl_Position  = vec4( 1.0, -1.0, 0.0, 1.0 );
  raster_color = vec3( 1.0, 0.0, blue );
  gl_Layer = gl_InvocationID; // Handle case where Layer Provoking vertex is Undefined
  EmitVertex();

  gl_Position  = vec4( 0.0, 1.0, 0.0, 1.0 );
  raster_color = vec3( 1.0, 1.0, blue );
  gl_Layer = gl_InvocationID; // Handle case where Last Vertex is Layer Provoking
  EmitVertex();

  EndPrimitive();
}

I would like to take this opportunity to point out that only 1 vertex in a primitive needs to have gl_Layer set; this vertex is called the Layer Provoking Vertex. Your shader assumes that the first vertex is the layer provoking vertex, but this is implementation-specific. When in doubt, the best solution is to cover all bases (set gl_Layer for all vertices).

You need to check GL_LAYER_PROVOKING_VERTEX at run-time to figure out which vertex defines your layer. If you do not want to do that, you can write your shader the way I described above. Provoking vertex conventions are usually first or last, but the way Geometry Shaders works leaves the possibility that any arbitrary vertex could define the layer (GL_UNDEFINED_VERTEX, and this is the case you should assume).

Answer 2

Turned out it was not a problem with gl_Layer. It was simply a syntax error in glTexParameter that was causing my resulting 3D texture to repeat rather than clamp to edges.