I have an application that is a fairly traditional "media player." It is implemented for Windows in C++17 with Qt5 providing the GUI.
One of the "key" features is a progress slider you can grab to scrub to any position in the video. (This is helpful for visually searching the video.) Unfortunately, the performance (that is, how often the QMediaPlayer actually updates the video display area while scrubbing) is much worse than desired.
Consider that, when playing a video, the update rate is smooth and usually between 15 and 60 Hz. When scrubbing, the video might only update at 2 or 3 Hz.
The video is loaded in a straight-forward way:
this->player.setMedia(QUrl::fromLocalFile(fileDialogResult->FilePath.string().c_str()));
The scrubbing is also done through a simple slot callback:
this->connect(this->sliderPlayback, &QSlider::sliderMoved, this, [this](auto position) {
this->blockSignals(true);
this->pimpl->player.play();
this->pimpl->player.setPosition(position);
this->pimpl->player.pause();
this->blockSignals(false);
});
To figure out where the bottleneck was, I ran the code through Intel VTune. It is actually spending all of its time (while scrubbing) down in the codecs. After more experimentation, I found that uncompressed (raw) videos actually performed really well. Compressed videos performed ok immediately after a keyframe, and then performance got worse as you moved away towards the next keyframe.
Is there a way to trade RAM for performance and playback a decoded version of the video every time? (i.e., pay the price for decoding up front, keep uncompressed frames in memory, and use those for playback?)
I tried loading the video and pushing it into a buffer, but this had no meaningful impact on performance.
QFile mediafile(fileDialogResult->FilePath.string().c_str());
mediafile.open(QIODevice::ReadOnly);
auto ba = new QByteArray();
ba->append(mediafile.readAll());
auto buffer = new QBuffer(ba);
buffer->open(QIODevice::ReadOnly);
buffer->reset();
this->player.setMedia(QMediaContent(), buffer);
I feel like this might be the right track, if the buffer contained the decoded (uncompressed) video. If there is a commercial library that I could license that would solve the problem, I would be happy to look into that as well.
