Zoom QMainwindow

Question

Short context:

I have a QMainwindow with an mpv player inside. I play videos with mpv, create overlay images with PIL and run it all in the pyqt window. The overlay image is updated more or less every frame of the video.

Here is my issue:

If the mpv picture is large, then updating the overlay images is far too slow (I have optimized a lot to improve performance, using separate processes and threads, only using one overlay etc.). If the picture is small, however, it all works flawlessly (thus indicating that it is not far from satisfactory in performance).

I wouldn't mind losing resolution to gain performance, so I want to have a large window with lower resolution content. Is this possible?

The bottleneck here is mpv's overlay.update function

My main idea is to zoom the QMainwindow, but I cannot seem to find a way to do this. Any other solution is of course sufficient.

Example code (note that test.mp4 is the hardcoded video, provide anything you have)

#!/usr/bin/env python3
import mpv
import sys

from PIL import Image, ImageDraw
from PyQt5.QtWidgets import *
from PyQt5.QtCore import *

class Test(QMainWindow):
    def __init__(self, parent=None):
        super().__init__(parent)
        self.container = QWidget(self)
        self.setCentralWidget(self.container)
        self.container.setAttribute(Qt.WA_DontCreateNativeAncestors)
        self.container.setAttribute(Qt.WA_NativeWindow)

        self.w = 1800
        self.h = int(self.w / 16 * 9)
        self.setFixedSize(self.w, self.h)
        
        self.player = mpv.MPV(wid=str(int(self.container.winId())),
                         log_handler=print)
        self.player.play('test.mp4')


        self.overlay = self.player.create_image_overlay()
        self.coords = [20, 20, 50, 50]

    def play(self):
        @self.player.property_observer("time-pos")
        def _time_observer(_name: str, value: float) -> None:
            
            for i in range(len(self.coords)):
                self.coords[i] = self.coords[i]*2 % self.h

            img = Image.new("RGBA", (self.w, self.h), (0, 0, 0, 0))
            draw = ImageDraw.Draw(img)
            draw.rectangle(self.coords, outline=(255,255,255,255), width=4)
            
            self.overlay.update(img)
            
            

app = QApplication(sys.argv)

# This is necessary since PyQT stomps over the locale settings needed by libmpv.
# This needs to happen after importing PyQT before creating the first mpv.MPV instance.
import locale
locale.setlocale(locale.LC_NUMERIC, 'C')
win = Test()
win.show()
win.play()

sys.exit(app.exec_())

Short Summary

Having a large window causes mpv's overlay.update method to consume too much time/computation. It is acceptable to decrease the dpi (resolution) of the overlay pictures, and even the video, to make it run faster.

From what I can see from the sources, it seems that `update` creates a new PIL.Image each time, maybe you can do some caching of the images (based on the current resolution) and directly use the mpv instance to call for `self.command('overlay_add`, ...)`. I cannot test this, and I cannot guarantee it will seriously improve the performance, as overlay objects are usually not intended to be animated. — musicamante, Sep 12 '20 at 17:35
The purp is the `to_bytes` call inside `overlay.update` . Creating a new image is actually fairly cheap. I considered cashing but it becomes quite complicated when frame-stepping and seeking in the video. — jkazan, Sep 12 '20 at 23:24
Unfortunately, with your simple example I cannot help you that much (I suppose you have something else in mind than random rectangles), but I'd suggest to start with a basic example creating a simple cache with a fixed set of images and see if there's an actual improvement. If that's so, you could create a custom object that return images based on the size and the timestamp, and manually call `instance.command('overlay_add', ...)`. — musicamante, Sep 13 '20 at 16:30
The actual answer to your question is that "zooming" of a widget is not possible (not like you want to, anyway) especially considering that the mpv object is actually a foreign window object embedded within the qt widget, and with video playing that also means that the underlying video system is possibly doing some software/hardware overlay you have no control over. The only alternative would be to avoid mpv at all and use [QMediaPlayer](https://doc.qt.io/qt-5/qmediaplayer.html) instead. — musicamante, Sep 13 '20 at 16:33
Actually, I pretty much want to draw rectangles, though not random, but I reckon the example should suffice. I'll edit the code in my question, with a larger window, and you'll see that it is no longer updating the rectangles fast enough. Caching could work, but is cumbersome when allowing seeking in the video and shouldn't be necessary since I am drawing very simple images. The problem seems so simple, yet the solution becomes overly complicated. — jkazan, Sep 14 '20 at 06:51
Another thought: If I implement caching, I need to run the caching procedure in a separate process, to utilize another cpu core. However, I had some problems sending the c_type char array from the caching process to the main process, where `overlay_add` is called. — jkazan, Sep 14 '20 at 07:15

Zoom QMainwindow

0 Answers0