More than one QGLWidget simultaneous doesn't work for a video usage

Question

I have a problem if I want to display a video using QGLWidget. With one instance it works, but it doesn't and widgets are black with multi-occurrence (usage in QGridLayout for example).

I subclass QGLWidget that way to play a video (by refreshing the QPixmap I want to show using my setCurrentImage method):

class GLWidget : public QGLWidget
{
    Q_OBJECT

public:
    GLWidget(QGLWidget* shareWidget = 0, QWidget* parent = 0)
        : QGLWidget(parent, shareWidget)
    {
        //...
        // I do this because the QPixmap to refresh is produced in a different thread than UI thread.
        connect(this, SIGNAL(updated()), this SLOT(update()));
    }

    //...

    void setCurrentImage(const QPixmap& pixmap)
    {
        m_mutex.lock();
        m_pixmap = pixmap;
        m_mutex.unlock();

        emit updated();
    }

protected:
    void initializeGL()
    {
        static const int coords[4][2] = {{ +1, -1 }, { -1, -1 }, { -1, +1 }, { +1, +1 }};
        for(int j = 0; j < 4; ++j){
            m_texCoords.append(QVector2D(j == 0 || j == 3, j == 0 || j == 1));
            m_vertices.append(QVector2D(0.5 * coords[j][0], 0.5 * coords[j][1]));
        }

        glEnable(GL_DEPTH_TEST);
        glEnable(GL_CULL_FACE);
        glEnable(GL_TEXTURE_2D);

#define PROGRAM_VERTEX_ATTRIBUTE 0
#define PROGRAM_TEXCOORD_ATTRIBUTE 1

        QGLShader* vShader = new QGLShader(QGLShader::Vertex, this);
        const char* szVShaderCode = /*...*/;
        vShader->compileSourceCode(szVShaderCode);

        QGLShader *fShader = new QGLShader(QGLShader::Fragment, this);
        const char* szFShaderCode = /*...*/;
        fShader->compileSourceCode(szFShaderCode);

        m_pProgram = new QGLShaderProgram(this);
        m_pProgram->addShader(vShader);
        m_pProgram->addShader(fShader);
        m_pProgram->bindAttributeLocation("vertex", PROGRAM_VERTEX_ATTRIBUTE);
        m_pProgram->bindAttributeLocation("texCoord", PROGRAM_TEXCOORD_ATTRIBUTE);
        m_pProgram->link();

        m_pProgram->bind();
        m_pProgram->setUniformValue("texture", 0);
    }

    void paintGL()
    {
        qglClearColor(Qt::black);
        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

        // I use some uniform values to "modify" image
        m_pProgram->setUniformValue(/*...*/);
        //...

        QMatrix4x4 m;
        m.ortho(-0.5f, +0.5f, +0.5f, -0.5f, 4.0f, 15.0f);
        m.translate(0.0f, 0.0f, -10.0f);

        m_pProgram->setUniformValue("matrix", m);
        m_pProgram->enableAttributeArray(PROGRAM_VERTEX_ATTRIBUTE);
        m_pProgram->enableAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE);
        m_pProgram->setAttributeArray(PROGRAM_VERTEX_ATTRIBUTE, m_vertices.constData());
        m_pProgram->setAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE, m_texCoords.constData());

        glBindTexture(GL_TEXTURE_2D, m_texture);
        glDrawArrays(GL_TRIANGLE_FAN, 0, 4);        
    }

    void resizeGL(int iWidth, int iHeight)
    {
        glViewport(0, 0, iWidth, iHeight);
    }

private slots:
    void update()
    {
        QPixmap pixmap;

        m_mutex.lock();
        pixmap = m_pixmap;
        m_mutex.unlock();

        m_texture = bindTexture(pixmap, GL_TEXTURE_2D);
        updateGL();
    }

private:
    QMutex m_mutex;
    QPixmap m_pixmap;

    QVector<QVector2D> m_vertices;
    QVector<QVector2D> m_texCoords;
    QGLShaderProgram* m_pProgram;
};

Then, to simulate a multi thread rendering (each thread produces its own video images), I wrote these little classes.

The image producer class:

class ImageProducer : public QThread
{
    Q_OBJECT

public:
    ImageProducer(QGLWidget* pGLWidget)
        : QThread(pGLWidget), m_pGLWidget(pGLWidget)
    {
        m_pixmap = QPixmap("fileName.jpg");
        m_bMustStop = false;
    }

protected:
    void run()
    {
        while(!m_bMustStop){
            static_cast<GLWidget*>(m_pGLWidget)->setCurrentImage(m_pixmap);

            // Simulate a frame rate
            msleep(1000 / /*FRAME_RATE*/);
        }
    }

private:
    QGLWidget* m_pGLWidget;
    QPixmap m_pixmap;
    bool m_bMustStop;
};

And the rendering class:

#define ROWS 1
#define COLS 1

class Window : public QWidget
{
public:
    Window()
    {
        QGridLayout* pMainLayout = new QGridLayout(this);
        setLayout(pMainLayout);

        for(int i = 0; i < ROWS; ++i){
            for(int j = 0; j < COLS; ++j){
                QGLWidget* pGLWidget = new GLWidget();
                pMainLayout->addWidget(pGLWidget, i, j);

                ImageProducer* pImageProducer = new ImageProducer(pGLWidget);
                pImageProducer->start();
            }
        }
    }
};

Ok stop with code samples ^^ Problem is with ROWS = 1 and COLS = 1 (see Window class) it works, but I have black widgets with other values... I'm lost, what do I miss?

Thanks!

EDIT: (context is always multi-GLWidget instance, all works fine with only one)

Strange thing I just discover: I overrode QGLWidget's mouseMoveEvent (so, in my GLWidget class) which simply calls updateGL();. And fact is when I press and move the mouse with the current code nothing happens. But if I replace (in my ImageProducer's run() method):

static_cast<GLWidget*>(m_pGLWidget)->setCurrentImage(m_pixmap);

By

static_cast<GLWidget*>(m_pGLWidget)->setCurrentImage(QPixmap("fileName.jpg"));

The image is refreshed in the current component as long as I move the mouse. When I release it, or do it in an other components, the background become black again.

Answer 1

I am writing down the answer after a chat with the OP, for the benefit of others. Here are the main topics that lead to a resolution of the issue.

Note: At the time of writing, the Qt OpenGL module is deprecated and its usage is discouraged. According to the official documentation the suggested approach is to use the OpenGL* classes in the GUI module.

Textures

The main issue with the code above is essentially due to the way the textures are handled.

First of all, the bindTexture() method does not bind in OpenGL terms, but actually create the OpenGL texture and upload the data passed as argument to it (QImage or QPixmap). That is confusing and can lead to serious issues. In the code posted by the OP, he is essentially leaking memory allocating a new texture at each frame update.

To ensure you are not leaking memory, you should at the least release the previously allocated texture, calling QGLWidget::deleteTexture.

However, it is worth to note that there are better approaches to avoid unnecessary memory fragmentation and inefficiency. In this case, the best approach is to allocate the texture once and simply update its content when necessary. This may not be possible directly with the API offered by the old Qt OpenGL module, but you can always mix Qt and native OpenGL code.

Context handling

Another issue is the way OpenGL context is handled. As a rule of thumb, one should always ensure the correct context is current before issuing commands to the OpenGL implementation. Sometimes, Qt does this for you automatically (i.e. before calling the paintGL() method).

In this case, we need to explicitly make the QGLWidget's context the current context before calling the bindTexture method, otherwise it will effect the last context that was made current. That is why only the last widget was showing something untill somewhone was triggering a makeCurrent call by interacting with the other widgets.

Threading

There are a couple of issues here. First of all, it is not safe to use a QPixmap object in a thread other than the GUI thread. QPixmap is designed to optimize pixmap blitting on screen. In this case, the pixmap is really just the frame to be uploaded to the OpenGL implementation, which handles all the rendering. So it is safe to use a QImage instead.

The other issue is that the GLWidget::setCurrentImage() and thus, the bindTexture method, are called directly from the run() method of the ImageProducer thread. This can't be because we need to make the widget context current, but it is not possible to call makeCurrent() from a thread other than the GUI thread (more details here).

A possible approach to solve this issue is to add a signal to ImageProducer to notify the frame has been updated, and connect this signal to the setCurrentImage() slot. Qt's signal-slot mechanism will ensure that the setCurrentImage is executed in the GLWidget thread, that is the GUI thread.

Code

Here are the suggested (and tested) modifications to the code posted above.

ImageProduer class

add the signal:

 void imageUpdated(const QImage &image);

and emit it when the frame is updated:

void ImageProducer::run()
{
    while(!m_bMustStop){
        QImage frame(m_pixmap.width(), m_pixmap.height(), m_pixmap.format());
        QPainter painter(&frame);
        painter.drawImage(QPoint(), m_pixmap);
        painter.setFont(QFont("sans-serif", 22));
        painter.setPen(Qt::white);
        painter.drawText(20, 50, QString("Frame: %1").arg(QString::number(_frameCount)));
        painter.end();

        emit imageUpdated(frame);

        msleep(1000 / FRAME_RATE);

        _frameCount++;
    }
}

GLWidget class

ensure the setCurrentImage method handle the OpenGL context and destroy the old texture:

void GLWidget::setCurrentImage(const QImage& pixmap)
{
    m_mutex.lock();
    m_pixmap = pixmap;
    m_mutex.unlock();

    makeCurrent();

    deleteTexture(m_texture);
    m_texture = bindTexture(pixmap, GL_TEXTURE_2D);

    doneCurrent();

    emit updated();
}