PyQt video frame update signal (Trigger function after each video frame)

Question

I am creating a video player and I need to draw some polygons on top of it. I am using a QGraphicsScene to create this and I need to update the polygons on screen after each frame. I am currently using the QMediaPlayer paired up with a QGraphicsVideoItem to create this. The problem I am having is that the QMediaPlayer doesn't have a signal that activates on each frame. It has positionChanged(), but this only seems to trigger once every second.

I tried using QMovie since it does send updates on every frame, but it did not display anything. This is the code I used to implement this.

    video_view = QGraphicsView()#view to hold video
    video_item = QGraphicsVideoItem()#video item for scene
    video_scene = QGraphicsScene()#scene for Qgraphics view
    video_view.setScene(video_scene)

    label = QLabel()
    movie = QMovie(self.video_paths[index]) #contains file path
    label.setMovie(movie)
    video_scene.addWidget(label)
    self.vlayout_main_video.addWidget(video_view)

The video file I am using is a .avi file and it is 72Mb large.

I would really appreciate it if somebody could point me in the right direction on how I could do this. I am currently using PyQt5.

Thank you

score 3 · Accepted Answer · answered Dec 12 '18 at 18:36

There are 2 options:

positionChanged is emited every second because the notifyInterval property of QMediaPlayer is set in that period. So you can change that property, for example to 60 ms.

from PyQt5 import QtCore, QtGui, QtWidgets, QtMultimedia, QtMultimediaWidgets

class MainWindow(QtWidgets.QMainWindow):
    def __init__(self, parent=None):
        super(MainWindow, self).__init__(parent)
        scene = QtWidgets.QGraphicsScene(self)
        self.video_view = QtWidgets.QGraphicsView(scene)
        self.setCentralWidget(self.video_view)

        self.player = QtMultimedia.QMediaPlayer(self, QtMultimedia.QMediaPlayer.VideoSurface)
        self.video_item = QtMultimediaWidgets.QGraphicsVideoItem()
        self.player.setVideoOutput(self.video_item)
        scene.addItem(self.video_item)
        file = "/path/of/video"
        self.player.setMedia(QtMultimedia.QMediaContent(QtCore.QUrl.fromLocalFile(file)))
        self.player.positionChanged.connect(self.on_positionChanged)
        self.player.setNotifyInterval(60)
        self.player.play()

    @QtCore.pyqtSlot('qint64')
    def on_positionChanged(self, p):
        print(p, QtCore.QTime.currentTime().toString("hh:mm:ss.zzz"))

if __name__ == '__main__':
    import sys
    app = QtWidgets.QApplication(sys.argv)
    w = MainWindow()
    w.resize(640, 480)
    w.show()
    sys.exit(app.exec_())

Use the VideoFrameProbed signal from QVideoProbe:

from PyQt5 import QtCore, QtGui, QtWidgets, QtMultimedia, QtMultimediaWidgets

class MainWindow(QtWidgets.QMainWindow):
    def __init__(self, parent=None):
        super(MainWindow, self).__init__(parent)
        scene = QtWidgets.QGraphicsScene(self)
        self.video_view = QtWidgets.QGraphicsView(scene)
        self.setCentralWidget(self.video_view)

        self.player = QtMultimedia.QMediaPlayer(self, QtMultimedia.QMediaPlayer.VideoSurface)
        self.video_item = QtMultimediaWidgets.QGraphicsVideoItem()
        self.player.setVideoOutput(self.video_item)
        scene.addItem(self.video_item)
        file = "/path/of/video"
        self.player.setMedia(QtMultimedia.QMediaContent(QtCore.QUrl.fromLocalFile(file)))
        self.player.play()

        probe = QtMultimedia.QVideoProbe(self)
        probe.videoFrameProbed.connect(self.on_videoFrameProbed)
        probe.setSource(self.player)

    @QtCore.pyqtSlot()
    def on_videoFrameProbed(self):
        print(QtCore.QTime.currentTime().toString("hh:mm:ss.zzz"))

if __name__ == '__main__':
    import sys
    app = QtWidgets.QApplication(sys.argv)
    w = MainWindow()
    w.resize(640, 480)
    w.show()
    sys.exit(app.exec_())

Thank you for the answer. It works great. Do you know if there is a way to check the index of the frame probed by on_videoFrameProbed? I have been going over the documentation and can't seem to find anything about this. — Miguel Kulisic, Dec 13 '18 at 01:58
@MiguelKulisic There is no such property but a possible solution is to create an index and increase it. — eyllanesc, Dec 13 '18 at 02:00
It seems strange to me but in my case it's never called on_videoFrameProbed.Is this example still working? — Fran Raga, May 14 '19 at 20:55
@FranRaga I have tested it now and it works correctly in Linux. Execute your application from the terminal or CMD and tell me if you get any message, on the other hand what is your OS? Did the first alternative work for you? — eyllanesc, May 14 '19 at 21:04
from terminal show `qt.multimedia.plugins.directshow: Unable to connect the video probe!`,interesting. I'm using windows 10 and QT version 5.11.2.Not work any example that uses QVideoProbe. I wanted to do a test on the order of calls, but I can't get this class to work.The video if playing — Fran Raga, May 14 '19 at 21:38
@FranRaga According to the [docs](https://doc.qt.io/qt-5/qtmultimedia-windows.html#limitations) Directshow has limitations and you will not be able to use QVideoProbe. *Video recording is currently not supported. Additionally, the DirectShow plugin does not support any low-level video functionality such as monitoring video frames being played or recorded using QVideoProbe or related classes.* — eyllanesc, May 14 '19 at 21:45
Thanks. I see that it seems a known problem ! https://forum.qt.io/topic/95260/unable-to-connect-the-video-probe-in-the-player-example — Fran Raga, May 14 '19 at 21:51

PyQt video frame update signal (Trigger function after each video frame)

1 Answers1