0

Short version: How can a crash (bad memory access exception/nullptr exception) inside glDrawElements/glDrawElementsInstanced be debugged?

Long version: You have a path of OpenGL rendering code which uses VAOs, and commits rendering thru calling glDrawElements or glDrawElementsInstanced. That codepath works correctly most of the time. We are talking "editor code", which means: the data may be any geometry and is highly likely to change frequently.

But sometimes after commiting reproducible data changes it simply crashes within the glDrawElements* driver code (i.e. glDrawElements is called, the function parameters are OK, the crash happens inside glDrawElements).

How could you proceed debugging this problem?

P.S.:

  • self-answered question: all research efforts went into the answer!
  • this is targeted at editor code. For simple demonstrations such crashes are mostly caused by the coder not understanding the requirements of glDrawElements correctly, and as such the codepath will either work, or not - in those cases, see:
Community
  • 1
  • 1
St0fF
  • 1,553
  • 12
  • 22
  • I don't understand why this question is downvoted all the time. Maybe there should be a notice right next to it, that it is self-answered? I know the question doesn't show research effort - because all research effort went into the answer! But it should still be clear, and it is indeed useful for everyone creating OpenGL Editors. – St0fF Dec 09 '16 at 09:31
  • 1
    Self-answering a question to share knowledge is fine. But the question must still be a *good question*. This is not. There's no way for anyone who *isn't you* to answer it. – Nicol Bolas Dec 09 '16 at 15:28
  • Well, how many people create integrated OpenGL-Graphics developing environments? I understand that this question and answer won't be useful to loads of people. But the few googling around may find it helpful. And yes, it is a black box magic "wth does it crash inside the driver" problem, that makes it hard to describe. I just tried to make it a little more clear, but I don't think it's gotten any better. – St0fF Dec 10 '16 at 21:09
  • P.S.: Regarding the fact that it works fine most of the time, but crashes on specific data modifications, it should be clear that there cannot be a "minimal, complete and verifiable example". – St0fF Dec 10 '16 at 21:12

1 Answers1

4

At first it should be clear what may cause crashes within the driver. In most cases that's bad memory access.

What may cause access of bad memory inside the driver?

  • GL_ELEMENT_ARRAY_BUFFER-binding changed somehow (thus the parameters given to glDrawElements may cause access beyond the memory of that object)
  • GL_ELEMENT_ARRAY_BUFFER's content has changed, possibly referencing uninitialized/non-existent vertex data (VBO access out of bounds)
  • any of the associated GL_ARRAY_BUFFER objects' data was altered, thus not containing referenced vertex data anymore (VBO access out of bounds)
  • or even more of such changes. Access violations inside glDrawElements* mostly mean that any of the objects bound within the VAO state was accessed out of bounds.

Without extra debugging code these access violations are very hard to catch. I suggest inserting debug-output right before calling glDrawElements*. The Debug output should query all the bindings and info available, so you can compare the settings "when it works" with "when it crashes" and figure out what to look for, next.

My debug function looks like this:

void debugVAOState(std::string baseMessage)
{
    baseMessage.append( " ... querying VAO state:\n" );
    int vab, eabb, eabbs, mva, isOn( 1 ), vaabb;
    glGetIntegerv( GL_VERTEX_ARRAY_BINDING, &vab );
    glGetIntegerv( GL_ELEMENT_ARRAY_BUFFER_BINDING, &eabb );
    glGetBufferParameteriv( GL_ELEMENT_ARRAY_BUFFER, GL_BUFFER_SIZE, &eabbs );

    baseMessage.append( "  VAO: " + std::to_string( vab ) + "\n" );
    baseMessage.append( "  IBO: " + std::to_string( eabb ) + ", size=" + std::to_string( eabbs )  + "\n" );

    glGetIntegerv( GL_MAX_VERTEX_ATTRIBS, &mva );
    for ( unsigned i = 0; i < mva; ++i )
    {
        glGetVertexAttribiv( i, GL_VERTEX_ATTRIB_ARRAY_ENABLED, &isOn );
        if ( isOn )
        {
            glGetVertexAttribiv( i, GL_VERTEX_ATTRIB_ARRAY_BUFFER_BINDING, &vaabb );
            baseMessage.append( "  attrib #" + std::to_string( i ) + ": VBO=" + std::to_string( vaabb ) + "\n" );
        }
    }
    OutputDebugString( baseMessage.c_str() );
}

It's still simple and outputs only the most valuable information to be able to see if above mentioned bindings have changed somehow. But this helped me finding numerous crashes that came from aggressive OpenGL optimization.

St0fF
  • 1,553
  • 12
  • 22