Receive webRTC video stream using python OpenCV in real-time

Question

Hi,
I am creating a pipeline where I need to access data from the camera and do some OpenCV algorithms in it. I am able to send the video from the source using webRTC. https://lostechies.com/derickbailey/2014/03/13/build-a-local-webcam-with-webrtc-in-less-than-20-lines/

But, What I need help with is how to receive the video stream in Python and do the processing. How can I access the video feed from a webRTC stream to the Python backend?

This is the javascript code running.

(function(){
  var mediaOptions = { audio: false, video: true };

  if (!navigator.getUserMedia) {
      navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
  }

  if (!navigator.getUserMedia){
    return alert('getUserMedia not supported in this browser.');
  }

  navigator.getUserMedia(mediaOptions, success, function(e) {
    console.log(e);
  });

  function success(stream){
    var video = document.querySelector("#player");
    video.src = window.URL.createObjectURL(stream);
  }
})();

I need help in receiving the video from this Javascript using Python.

score 17 · Accepted Answer · answered Dec 27 '18 at 16:34

17

I'm the author of aiortc. Have you checked out the server example, as it illustrates how to process video using OpenCV?

https://github.com/jlaine/aiortc/tree/master/examples/server

answered Dec 27 '18 at 16:34

Jeremy

1,308
10
11

I tried installing aiortc in Ubuntu 14.04 and was not able. Installed it in ubuntu 16.04 and works well to my use case. Thank you so much!!! – Sreekiran A R Dec 31 '18 at 06:07
This is a good example to process video from the own device. However, how do you let a foreign device send the video to this server? – WJA Jul 22 '19 at 21:15
1

@jeremy is it correct that there is no Windows support? – WJA Jul 23 '19 at 22:49
There is nothing to prevent you from using aiortc on Windows, but it involves compiling ffmpeg / libvpx / libopus so it's probably tedious – Jeremy Jul 26 '19 at 17:10
1

@JohnAndrews Were you able to process video from a foreign device? – Anil Oct 02 '19 at 23:23
1

I can NEVER use this at all for some reasons. Connected with no media flow :( ...! – Thư Sinh Nov 05 '21 at 10:55

score 3 · Answer 2 · answered Nov 07 '18 at 13:37

3

https://webrtchacks.com/webrtc-cv-tensorflow/ shows a fairly in-depth tutorial for doing WebRTC + tensorflow. You can probably swap out tensorflow for opencv easily. This captures a frame from the webcam and sends it using HTTP every once in a while. If you want to go more realtime than that you will have to use WebRTC on the server, e.g. using https://github.com/jlaine/aiortc

answered Nov 07 '18 at 13:37

Philipp Hancke

15,855
2
23
31

2

I need it in real-time. I had went through this aiortc but the receiving part is not clear in the documentation. Have you tried it? – Sreekiran A R Nov 08 '18 at 05:25
This only uses the media capture APIs but it's not using RTCPeerConnection anywhere. – Jack Vial Nov 14 '20 at 18:59

Receive webRTC video stream using python OpenCV in real-time

2 Answers2

Linked