GL_INVALID_OPERATION on Tesselation Shader

Question

I'm using OpenGL with LWJGL and my own very small framework for trivial tasks. I'm following the book OpenGL SuperBible: Comprehensive Tutorial and Reference (6th Edition).

I will list the most important parts of my program:

public class GameController extends Controller {
    private Program test1Program;
    private int vaoId;

    @Override
    protected void init() {
        glViewport(0, 0, 800, 600);
        test1Program = new Program(
                new VertexShader("src/shaders/test.vert.glsl").create(),
                new ControlShader("src/shaders/test.cont.glsl").create(),
                new EvaluationShader("src/shaders/test.eval.glsl").create(),
                new FragmentShader("src/shaders/test.frag.glsl").create()
                ).create();
        vaoId = glGenVertexArrays();
        glBindVertexArray(vaoId);
    }

    @Override
    protected void draw(double msDelta) {
        glClearColor((float)Math.sin(currentTime / 1000f) * 0.5f + 0.5f, (float)Math.cos(currentTime / 1000f) * 0.5f + 0.5f, 0.0f, 1.0f);
        glClear(GL_COLOR_BUFFER_BIT);        
        test1Program.use();
        glVertexAttrib4f(0, (float)Math.sin(currentTime / 1000f) * 0.5f, (float)Math.cos(currentTime / 1000f) * 0.5f, 0.0f, 0.0f);
        glVertexAttrib4f(1, (float)Math.sin(currentTime / 1000f * 2f) * 0.5f + 0.5f, (float)Math.cos(currentTime / 1000f * 2f) * 0.5f + 0.5f, 0.0f, 1.0f);
        glPolygonMode(GL_FRONT_AND_BACK, GL_LINE);
        glDrawArrays(GL_TRIANGLES, 0, 3);
    }

    @Override
    protected void shutdown() {
        test1Program.delete();
        glDeleteVertexArrays(vaoId);
    }

    public static void main(String[] args) {
        Controller controller = new GameController();
        controller.start();
    }
}

My custom VertexShader, ControlShader, EvaluationShader and FragmentShader are working and if shader code does not compile properly or does not get linked correctly, then an exception will be thrown and I would notice it. So I have verified that those and Program are working correctly.

The error (Exception in thread "main" org.lwjgl.opengl.OpenGLException: Invalid operation (1282)) gets thrown at the glDrawArrays call.

test.vert.glsl:

#version 440 core

layout(location = 0) in vec4 offset;
layout(location = 1) in vec4 color;

out VS_OUT {
    vec4 color;
} vs_out;

void main() {
    const vec4 vertices[3] = vec4[3](
        vec4(0.25, -0.25, 0.5, 1.0),
        vec4(-0.25, -0.25, 0.5, 1.0),
        vec4(0.25, 0.25, 0.5, 1.0)
    );
    gl_Position = vertices[gl_VertexID] + offset;
    vs_out.color = color;
}

test.cont.glsl:

#version 440 core

layout(vertices = 3) out;

void main() {
    if (gl_InvocationID == 0) {
        gl_TessLevelInner[0] = 5.0;
        gl_TessLevelOuter[0] = 5.0;
        gl_TessLevelOuter[1] = 5.0;
        gl_TessLevelOuter[2] = 5.0;
    }
    gl_out[gl_InvocationID].gl_Position = gl_in[gl_InvocationID].gl_Position;
}

test.eval.glsl:

#version 440 core

layout(triangles, equal_spacing, cw) in;

void main() {
    gl_Position = (gl_TessCoord.x * gl_in[0].gl_Position + gl_TessCoord.y * gl_in[1].gl_Position + gl_TessCoord.z * gl_in[2].gl_Position);
}

#version 440 core

in VS_OUT {
    vec4 color;
} fs_in;

out vec4 color;

void main() {
    color = fs_in.color;
}

I have triple checked all my code and crosschecked with the book, but have no clue why it is not working. I would appreciate any help and am around to provide additional information if neccessary.

To that downvoter: Give a **reason** for the downvote. To which part of the posting standards do I not adhere? — skiwi, Jan 14 '14 at 20:19
Does the code work with the default pipeline (i.e. without test1Program)? Also just to be safe bind the VAO again before rendering it. — Njol, Jan 14 '14 at 20:24
@Njol The code worked with only a `VertexShader` and a `FragmentShader`. Just verified that it runs with the default pipeline aswell. — skiwi, Jan 14 '14 at 20:28
More than likely the issue is related to not using a VAO in a core context... usually `GL_INVALID_OPERATION` is raised in the context of a GLSL program long before you draw anything with it (i.e. when you call `glUseProgram (...)`). It is perfectly acceptable to draw with no vertex pointers, but you still need to bind a VAO before you can make a call to `glDrawArrays (...)` in a core context. — Andon M. Coleman, Jan 14 '14 at 20:31
@AndonM.Coleman Working in `core=false` context in both LWJGL and the shaders did not change anything. I have also bound to an (empty) VAO before doing `glDrawArrays`. — skiwi, Jan 14 '14 at 20:36

score 7 · Accepted Answer · answered Jan 14 '14 at 20:49

7

I found the answer just now:

    glDrawArrays(GL_TRIANGLES, 0, 3);

needs to be:

    glDrawArrays(GL_PATCHES, 0, 3);

Some more effort from OpenGL to show what was wrong would have been appreciated.

Also nowhere in the book it has been (explicitely) mentioned that I need to use GL_PATCHES, I just figured it out by looking at the source code of the compilable examples from the book.

answered Jan 14 '14 at 20:49

skiwi

66,971
31
131
216

Ran into the same problem. Thanks for sharing. I had other problems, though. I had to ignore the interface block that carries the colour data around and hard-code a value in the fragment shader for it to display anything at all. I hope I'll understand why further down the line. – cangrejo Jun 28 '14 at 17:26

score 0 · Answer 2 · edited May 23 '17 at 11:58

After a more thorough investigation of your shaders, I suspect part of the problem is that there is no vertex attribute 1 (layout(location = 1) in vec4 color) in your program after all of the stages are linked. It is probably not your entire problem, but as it stands right now your GLSL program will not behave the way you want.

You have to pass the data fed into the vertex shader through the tessellation shader stages to get it into the fragment shader stage - if you do not do this, then the GLSL compiler/linker determines that that vertex attribute is not used when the program executes and does not assign it a location. This behavior applies to uniforms as well.

Have a look at another answer I wrote on SO for an explanation on how to do this.

GL_INVALID_OPERATION on Tesselation Shader

2 Answers2