1

I am trying to transfer a stream using Opencv using VideoWriter (c++ side using WSL) to get it on another pc on the network using VideoCapture (Unity side).

Inspired by this example: https://opencv94.rssing.com/chan-61447116/article1201-live.html

I want to send a video from OpenCV from C++ to Unity.

So I have this code on the c++ side:

cv::VideoWriter writer;
    writer.open("appsrc ! videoconvert ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast ! rtph264pay ! udpsink host=127.0.0.1 port=5000",
                        cv::CAP_GSTREAMER, 
                        0, 
                        5,
                        cv::Size (640, 480),
                        true);

When I run the c++ program it works and I'm able to use the writer.

on the unity side I have this:

VideoCapture capture = new VideoCapture();
bool opened = capture.open("udpsrc port=5000 ! rtph264pay ! decodebin ! videoconvert ! appsink sync=false", Videoio.CAP_GSTREAMER);

But I always get false on opened so I cant' connect to the c++.

I think the pipelines can be wrong in this case.

Is it really possible to receive a video from gstreamer on Unity?

Note: the c++ side was made using WSL ubuntu 18.06

felipe
  • 1,212
  • 1
  • 15
  • 27
  • I've achieved gstreamer to unity side, result in this post. https://forum.unity.com/threads/how-to-deploy-gstreamer-unity-to-oculus-quest2.1221267/#post-8344683 – sharimken Aug 07 '22 at 23:46

1 Answers1

0

Main cause may be that your sender is streaming to port 5000 while your receiver tries to read from port 5200. Assuming unity is the same host as writer, you would try:

Sender:

writer.open("appsrc ! videoconvert ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast insert-vui=1 ! h264parse ! rtph264pay config-interval=1 ! udpsink host=127.0.0.1 port=5000 auto-multicast=0",
            cv::CAP_GSTREAMER, 
            0, 
            5.f,
            cv::Size (640, 480),
            true);

Unity receiver:

[EDIT: user @felipe finally reported that OpenCVForUnity is not supported on Unity today 25/mar/2022]

bool opened = capture.open("udpsrc port=5000 ! application/x-rtp,encoding-name=H264 ! rtph264pay ! decodebin ! videoconvert ! appsink sync=false", Videoio.CAP_GSTREAMER);
SeB
  • 1,159
  • 6
  • 17
  • you saw it properly, but it wasn't this because I typed wrongly. So this is not the issue with the code. thank you – felipe Mar 12 '22 at 00:33
  • I also added some changes in both pipelines, you may give them a try. I have no experience with opencv in Unity, though, so just modified the pipeline. – SeB Mar 12 '22 at 00:52
  • It didn't worked :-(, if I have on sender: udpsink host=192.168.0.13 port=5000 auto-multicast=0 e.g. how can I get receiver: udpsrc on the port 192.168.0.13 for example? – felipe Mar 12 '22 at 01:18
  • Then host with IP 192.68.0.13 would be the only host receiving the stream, and it could be opened with a gstreamer pipeline starting by `udpsrc port=5000 ! ...`. Also try `insert-vui` for encoder and `config-interval=1` for `h264parse` or `rtph264pay`. Be aware that it may take some seconds to setup. Also, can you open with Unity a test pipeline such as `videotestsrc ! videoconvert ! appsink` ? – SeB Mar 12 '22 at 01:31
  • the videotestsrc pipeline didn't work. About 192.68.0.13, if this is in another pc how can the receiver udpsrc port=5000 receives the packets? shouldn't I point the receiver to 192.168.0.13 ? – felipe Mar 12 '22 at 01:44
  • The receiver is pointed by the sender using host property. First have opencv work within Unity before trying to receive stream. – SeB Mar 12 '22 at 01:46
  • Hi @SeB should I have a gstreamer host like a server app running on the host, or normally just the client capture.open("udpsrc... should be enough to open this host? – felipe Mar 14 '22 at 12:12
  • Hi @felipe, are you able to display the video from videotestsrc on receiver ? If not first try to get this. Note that opencv appsink would mainly expect BGR format for color frames so you would try this pipeline: `videotestsrc ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1`. I feel a bit surprized by the backend arg `Videoio.CAP_GSTREAMER`. Is it C code ? When you have the gstreamer capture backend working on receiver then you can try to receive stream. You don't need any additional server on receiver side. Be sure that no firewall rule blocks the stream. – SeB Mar 15 '22 at 20:20
  • Hi @SeB thank you for all the replies. It's Unity c#, and I think the problem is related to OpenCVForUnity that doesn't have pre-built Gstream by default. I had to compile it separately I think it's not a pipeline problem, because I could create sender/receiver with windows opencv compiled. So I on Unity side I'm getting not loading dlls problems now I posted here: https://forum.unity.com/threads/released-opencv-for-unity.277080/page-57#post-7965246 – felipe Mar 15 '22 at 20:28
  • Opencv provides a function getBuildInformation() that returns a string with the config, so you would be able to check if it was built with GSTREAMER_SUPPORT or not. Sorry I have no experience with this case and cannot help much more. – SeB Mar 15 '22 at 20:30
  • Thank you @SeB for all the responses, the final result is that: OpenCVForUnity is not supported on Unity today 25/mar/2022. I could run it on windows but not on android. – felipe Mar 25 '22 at 11:35