Converting from VlcMedia to Mat and vice versa

Question

I'm trying to connect "vlc-qt" to "opencv". I'm reading my ip camera using VlcInstance and VlcMedia.

As far as I know for OpenCV to work , it need to put the video in Mat format:

videoCapture cap;
Mat frame ;
cap >> frame ;

but I have it in VlcMedia format. Is there a way to convert that to Mat format (or as a source to videocapture) and then converting it back to VlcMedia to be able to show it in VlcMediaPlayer.

I need it to be fast (actually realtime fast :D).

This is my code (I'm using Qt in Visual Studio):

My header file:

#ifndef QT_VLC_OPENCV1_H
#define QT_VLC_OPENCV1_H

#include <QtWidgets/QMainWindow>
#include "ui_qt_vlc_opencv1.h"

#include <VLCQtCore\Instance.h>
#include <VLCQtCore\Media.h>
#include <VLCQtCore\MediaPlayer.h>

class qt_vlc_opencv1 : public QMainWindow
{
    Q_OBJECT

public:
    qt_vlc_opencv1(QWidget *parent = 0);
    ~qt_vlc_opencv1();
private slots:
    void on_btnOpenFile_clicked();

private:
    Ui::qt_vlc_opencv1Class ui;

    VlcInstance *_instance;
    VlcMedia *_media;
    VlcMediaPlayer *_player;
};

#endif // QT_VLC_OPENCV1_H

and my cpp file :

#include "qt_vlc_opencv1.h"

#include <VLCQtCore\Common.h>
#include <qinputdialog.h>

#include <opencv2/highgui/highgui.hpp>
#include <cv.h>
using namespace cv;

qt_vlc_opencv1::qt_vlc_opencv1(QWidget *parent)
: QMainWindow(parent), _media(0)
{
    ui.setupUi(this);

    _instance = new VlcInstance(VlcCommon::args(), this);
    _player = new VlcMediaPlayer(_instance);
    _player->setVideoWidget(ui.video);

    ui.video->setMediaPlayer(_player);
}

qt_vlc_opencv1::~qt_vlc_opencv1()
{
    delete _player;
    delete _media;
    delete _instance;
}

void qt_vlc_opencv1::on_btnOpenFile_clicked()
{
    QString file = QInputDialog::getText(this,tr("Open File"),tr("Enter a file path or a Url"));
    if (file.isEmpty())
        return;

    _media = new VlcMedia(file, true, _instance);
    _player->open(_media);
}

probably somewhere you can get the `QImage` of the current frame, then you can convert between `QImage` and `Mat`. — Miki, Sep 22 '15 at 13:16
tnx for reply . that's the question . how can i get the image from VLcMedia. — user2798392, Sep 22 '15 at 17:28

score 0 · Answer 1 · answered Jul 26 '19 at 10:35

in Qt-vlc library there is a VlcVideoStream class. VlcVideoStream sets proper callbacks to get YUV frames from libVLC. This class should be subclassed and implement frameUpdated to specify what to do with the frame. in the code below frameUpdator is a subclass of VlcVideoStream that i have impelemented its frameUpdated function. for example you can implement frameUpdated in this way :

  void frameUpdator::frameUpdated()
 {
int rows, cols, matType;

// convert to shared pointer to const frame to avoid crash
std::shared_ptr<const VlcYUVVideoFrame> frame = std::dynamic_pointer_cast<const VlcYUVVideoFrame>(renderFrame());

if (!frame) {
    return; // LCOV_EXCL_LINE
}

 //     rows = frame->height + frame->height / 2;
 //     cols = frame->width;
 //     matType = CV_8UC1;
 // 
 //     cv::Mat myuv(rows, cols, matType, (void*)frame->frameBuffer.data());
 //     cv::Mat mrgb(frame->height, frame->width, CV_8UC3);
 //     cv::cvtColor(myuv, mrgb, CV_YUV2RGB_I420);

  int width = frame->width;
  int height = frame->height;

   cv::Mat result = cv::Mat(height, width, CV_8UC1,(void*)frame->frameBuffer.data());
cv::imshow("result", result);
cv::waitKey(2);

   }

Converting from VlcMedia to Mat and vice versa

1 Answers1