OpenGL crashes on virtual machine (Ubuntu 16.04) setuped on VMWare Fusion Version 10.1.3.

Question

I'm trying to draw some graphics using OpenGL. I'm using VMWare Fusion Version 10.1.3 with Ubuntu 16.04 and QtCreator 4.6.2.

My task is to play recording with some drawing in OpenGL. In case of that, I need FBO to increment the content of my drawing and create an image on the end of the recording and I also need texture to display on screen during playing the recording.

My problem is that I have a permanent crash during destroying an object in which I use OpenGL.

Here is my code:

void DrawingView::paint(QPainter *painter) {

    painter->beginNativePainting();
    if(_needsErase) {

        QOpenGLContext  *context = QOpenGLContext::currentContext();

        _frameBuffer = new QOpenGLFramebufferObject(QSize(_rect.width(), _rect.height()), _format);

        glBindTexture(GL_TEXTURE_2D, _frameBuffer->texture());
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);

        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

        glTexParameteri(GL_TEXTURE_2D, GL_GENERATE_MIPMAP, GL_FALSE);

        glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, _rect.width(), _rect.height(), 0,  GL_RGBA , GL_UNSIGNED_BYTE, 0);

        glBindTexture(GL_TEXTURE_2D, 0);

        _frameBuffer->bind();
        {
            glClearColor(0.0, 0.0, 0.0, 0.0);
            glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
        }
        _frameBuffer->release();

        _needsErase = false;
    }

    glEnable(GL_BLEND);

    _frameBuffer->bind();
    { // here I'm drawing to my fbo, this part is irrelevant. }
    _frameBuffer->release();

    _shaderProgram->release();

    glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);

    glUniform1i(_tex0Uniform, 0);
    _copyBuffer->bind();

    glActiveTexture(GL_TEXTURE0);
    glEnable(GL_TEXTURE_2D);
    glBindTexture(GL_TEXTURE_2D, _frameBuffer->texture());

    glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, squareVertices);
    glEnableVertexAttribArray(ATTRIB_VERTEX);
    glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

    glActiveTexture(GL_TEXTURE0);
    glBindTexture( GL_TEXTURE_2D, 0);
    glDisable(GL_TEXTURE_2D);

    _copyBuffer->release();

    glDisable(GL_BLEND);

    painter->endNativePainting();
}

App crashes during destroying my DrawingView Object after line delete _frameBuffer; with log:

context mismatch in svga_surface_destroy
VMware: vmw_ioctl_command error Invalid argument.

Here is how I'm freeing my DrawingView Object.

void DrawingView::cleanupGL() {
    glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
    glClear(GL_COLOR_BUFFER_BIT);

    _shaderProgram->release();
    _frameBuffer->release();
    _copyBuffer->release();

    delete _shaderProgram;
    delete _copyBuffer;
    delete _frameBuffer;

    glDeleteBuffers(1, &_ebo);
    glDeleteBuffers(1, &_vbo);
}

I noticed that app stops crashing when I erase line with glDrawArrays(...) command, but I need this command to display drawing during playing the recording.

I was using this code also on macOS and there everything was working PERFECTLY, but unfortunately, I need to use this code on a virtual machine with Ubuntu.

Here is also the code of my vertex and fragment shaders:

Vertex shader:

uniform vec2 screenSize;
attribute vec2 position;
varying vec2 coord;

void main(void)
{
    coord = position;
    vec2 halfScreenSize = screenSize * 0.5f;
    vec2 pos = halfScreenSize * position + halfScreenSize;
    gl_Position = gl_ModelViewProjectionMatrix * vec4(pos, 0.0, 1.0);
}

Fragment shader:

uniform sampler2D tex0;
varying vec2 coord;

void main(void)
{
    vec2 coords = coord * 0.5 + 0.5;
    gl_FragColor = texture2D(tex0, coords.xy);
}

Do anyone have any idea why this doesn't want to work on a virtual machine?

I really wanted to be specific describing my problem, but if You have any further questions please ask.

Answer 1

You create a (new) QOpenGLFramebufferObject instance only if _needsErase ever is set true.

if(_needsErase) {
        QOpenGLContext  *context = QOpenGLContext::currentContext();
        _frameBuffer = new QOpenGLFramebufferObject(…

If that doesn't happen, it will be an uninitialized pointer, and calling member functions on it will invoke undefined behaviour. This particular code it wrong anyway, because it never deletes whatever instance may have been pointed to by _frameBuffer before overwriting the pointer.

Instead of manual memory management I strongly advise, to make use of automatic constructs. I appreciate the need for dynamic instance creation in this particular case. The way to go about this is through either std::shared_ptr or std::unique_ptr. I strongly suggest to start with std::unique_ptr and change it to a shared pointer only if absolutely needed so.

In your DrawingView class

#include <memory>
class DrawingView : public ... {
    ...
protected:
    std::unique_ptr<QOpenGLFramebufferObject> _frameBuffer;
    ...
};

in the DrawingView::paint method:

 if(_needsErase) {
         QOpenGLContext  *context = QOpenGLContext::currentContext();
         _frameBuffer = std::make_unique<QOpenGLFramebufferObject>(…

Do not use new or delete.