intermittent GLSL vertex shader compile errors

Question

I've been getting intermittent errors compiling a vertex shader, preparing for the first render of a newly created OpenGL context. It's the same vertex shader that usually works, on the same hardware. After the failure, the info log returned by glGetShaderInfoLog usually shows this:

Vertex shader failed to compile with the following errors:

Yes, that's the whole thing. It happens on Windows, with both ATI and NVidia GPUs, though I've mostly been testing on ATI. If I'm running under the Visual Studio debugger, an access violation may be detected in glCompileShader or in the subsequent glGetShaderiv call.

I haven't seen the bug on the Mac, but since it's not always easy to reproduce it, I'm not absolutely sure it doesn't happen.

I have noticed that if I don't call wglShareLists when I create my contexts, the error goes away (or at least I can't reproduce it easily). I take that to mean that some bad information is bleeding through from a previously created (and perhaps previously destroyed) OpenGL context to the new one. However, I've been trimming out stuff until there is not much that should be shared: No texture objects, display lists, VBOs, PBOs. The only things left that would be shared, as far as I know, would be shader and program objects.

After creating an OpenGL context for a window, I initialize shading roughly like this:

// Program initialization
GLint vertexShader = glCreateShader( GL_VERTEX_SHADER );
glShaderSource( vertexShader, 1, &vertexShaderSource, NULL );
glCompileShader( vertexShader );
GLint   status;
glGetShaderiv( vertexShader, GL_COMPILE_STATUS, &status );
if (status == GL_TRUE)
{
    GLint fragmentShader = glCreateShader( GL_FRAGMENT_SHADER );
    glShaderSource( fragmentShader, 1, &fragShaderSource, NULL );
    glCompileShader( fragmentShader );
    glGetShaderiv( vertexShader, GL_COMPILE_STATUS, &status );
    if (status == GL_TRUE)
    {
        GLint program = glCreateProgram();
        if (program != 0)
        {
            glAttachShader( program, vertexShader );
            glAttachShader( program, fragmentShader );
            glLinkProgram( program );
            glDetachShader( program, fragmentShader );
            glDetachShader( program, vertexShader );
            glDeleteShader( fragmentShader );
            // check for link success...
        }
    }
}
else
{
    char logBuf[1024];
    glGetShaderInfoLog( vertexShader, sizeof(logBuf), NULL, (GLchar*)logBuf );
    LogToFile( (char*)logBuf );
}

Then I render multiple frames, and at some point clean up

glDeleteProgram( program );
glDeleteShader( vertexShader );

and destroy the OpenGL context. Later, I create a new OpenGL context in the same window, initialize it in the same way, but the vertex shader fails to compile.

If I don't render any geometry, I can't get the bug to reproduce, but rendering a single point suffices.

It still happens with a simplified vertex shader:

#version 120
void main()
{
    gl_Position = ftransform();
    gl_FrontColor = gl_Color;
    gl_BackColor = gl_Color;
}

and a simplified fragment shader:

#version 120
void main()
{
    gl_FragColor = gl_Color;
}

I tried AMD's CodeXL OpenGL debugger, setting it to break on errors. It just tells me that the compile is failing, which I already knew.

The OpenGL context that fails is on a window where a different context was previously created, successfully rendered, and destroyed. There's nothing wrong with re-using a window like that, is there? (I am aware that once you've called SetPixelFormat, you can't change the pixel format of the window. I made sure that I'm using the same pixel format each time.)

ADDED: While trimming down the rendering code, I found that if I commented out the line

glEnable( GL_VERTEX_PROGRAM_TWO_SIDE );

then the error went away. But with more real-world rendering (e.g., textures) reinstated, it doesn't seem to make a difference.

ADDED 3/7/2014: I finally made a self-contained Visual C++ project that can reproduce the bug under certain circumstances, so that any interested parties can try it: ShaderBugTest project To make the error reproducible, you need to have Microsoft's "Application Verifier" set to watch for Heap errors. An enclosed Read Me has more detailed steps to reproduce.

Answer 1

You're example has a lot of allocation and deallocation of GL objects, but little to no error checking.

vertexShader = glCreateShader( GL_VERTEX_SHADER );

Need to check the return value, vertexShader, to make sure it's valid.

glGetShaderiv( vertexShader, GL_COMPILE_STATUS, &status );

You get this value, but don't show that you're checking status.

program = glCreateProgram();

Again, need to check the return value before you start using it.

glAttachShader( program, vertexShader );
fragmentShader = glCreateShader( GL_FRAGMENT_SHADER );
glShaderSource( fragmentShader, 1, &fragShaderSource, NULL );
glCompileShader( fragmentShader );

You should create your fragment shader and vertex shader both before you attach them to the program. It gives you less to clean up if something fails.

 glGetShaderiv( fragmentShader, GL_COMPILE_STATUS, &status );

Again, check the compile status after fetching it. You compile is failing, but so far we don't know which shader is failing to compile. Is it the fragment shader or the vertex shader.

 glAttachShader( program, fragmentShader );
 glDeleteShader( fragmentShader );

Deleting your shader here is weird. Yes, it should be only flagged for deleting, since it's actually attached to the program, but this still seems like a dangerous way of doing this. Also, you're taking a different approach to managing the fragment shader than the vertex shader, for no apparent reason.

 glLinkProgram( program );
 glDetachShader( program, fragmentShader );
 glDetachShader( program, vertexShader );

Again, this is a somewhat baffling approach to shader management to me. And again, treat the fragment shader and vertex shader similarly.

 glDeleteProgram( program );
 glDeleteShader( vertexShader );

Why the different treatment of the fragment shader?

While you're diagnosing the problem, you should be using glGetError() and stop the program if it reports anything out of the ordinary after every GL call so that you can isolate where the first error occurs. It's entirely possible that the compile failure is a cascade effect from something that happened previously.

Answer 2

Use glGetShaderInfoLog to see the compile error message. For an example, see http://www.opengl.org/wiki/Shader_Compilation#Shader_error_handling

Answer 3

It looks like you are not writing anything to gl_Color.

gl_Color attribute can mean different things in different places.

In your vertex shader, gl_Color represents the per-vertex color attribute passed by the user using glColor* or glColorPointer calls.

In your fragment shader, gl_Color represents the interpolated color for the facing of the triangle being rendered.

However, this is a fixed-pipeline functionality and gl_Color was deprecated in the modern OpenGL. This may be the reason behind your error. You need to get the compiler error message using the following code:

GLint   SuccessFlag = 0;
GLsizei Length      = 0;
GLsizei MaxLength   = 0;

glGetShaderiv( ShaderID, GL_COMPILE_STATUS,  &SuccessFlag );
glGetShaderiv( ShaderID, GL_INFO_LOG_LENGTH, &MaxLength   );

char* Log = ( char* )alloca( MaxLength );

glGetShaderInfoLog( ShaderID, MaxLength, &Length, Log );

Adding the log to your question will help to diagnose your problem better.

What version of OpenGL do you use? Is it core or compatibility profile?