1

There is a lot of information scattered over the internet, but it is hard to find something concrete

The question is what is the best modern way to create a live, low latency solution for video streaming

What solutions are being used for products like Wyze cams, Ring video doorbells, etc.

What are the best solutions for Android/ios clients and for web clients

Guy Levin
  • 1,230
  • 1
  • 10
  • 22
  • 1
    This is a very broad question. Can you hint what you want to do and what you want to avoid? Do you want to build a conference app, where multiple users talk to each other in a room? Are you able to run one or more cpu intensive media servers? Is security a top priority, because then you may even prefer a mesh based architecture, since this is the only way to offer end-to-end encryption – nasskalte.juni Oct 20 '19 at 11:33
  • Let's say for the conversation, that I want to build a security cam, with very low latency, two way audio, and security is an issue. I prefer not to have a cpu intensive server to encode videos, tough it is an option to have some server – Guy Levin Oct 20 '19 at 11:51
  • 2
    Ok, I imagine you stream the security cam video to one or a handful of other peers, but probably not thousands of other receivers. Then you may want to just use the standard approach without media servers (which break end 2 end encryption and would be a possible attack target). Depending on how secure it must be, have a look at [SaltyRTC](https://saltyrtc.org/), developed for Threema. It is a signalling server solution that offers secure signalling, even if someone hacked the signalling server. – nasskalte.juni Oct 20 '19 at 11:59
  • Thank you, so you are suggesting to deploy a WbbRTC server, and WebRTC clients to establish a peer to peer connection. What would you use to stream the video? And what to stream the 2 way audio? – Guy Levin Oct 20 '19 at 12:12
  • 2
    Streaming 2 way audio and one way video is part of webrtc. I don't know if you have already worked with the WebRTC-API, but adding a video-stream is quite easy. You can just call `myRTCPeerConnection.addTransceiver(someVideotrack, {direction: "sendonly"})` or for the audio `myRTCPeerConnection.addTransceiver(someVideotrack, {direction: "sendrecv"})`. The webrtc api takes care of sending the stream over an srtp connection, but setting up signalling (forwarding messages to build a pure Peer-to-Peer-Connection) can be tedious. – nasskalte.juni Oct 20 '19 at 12:30
  • Thank you. Why would you recommend SaltyRTC over other implemetations? – Guy Levin Oct 20 '19 at 12:38
  • 1
    As far as I am concerned, the only way to Man-in-the-middle attack a WebRTC call is by gaining control over the signalling server. SaltyRTC is the only Signalling-Server-Framework or -Implementation that tackles this problem, I simply do not know other implementations that do this. – nasskalte.juni Oct 20 '19 at 13:06
  • Thank you very much nasskalte.juni for your answers – Guy Levin Oct 20 '19 at 13:09
  • SaltyRTC doesn't look like it's highly used and highly maintained, are there any other recommendations? Maybe something that is also easy to integrate with tensorflow/pytorch – Guy Levin Oct 20 '19 at 22:36
  • Hard to say, I am nearly at my wits end. In my experience, working with webrtc libraries made things initially easier, but for everything more complex, the code quickly became ugly or what I wanted to do was not doable. If you use a signalling server or webrtc library, check for recent updates, since the standard is quite new and code tends to be deprecated (still using addStream instead of the newer Transceiver-API or similar things). – nasskalte.juni Oct 20 '19 at 23:44

1 Answers1

0

The "low latency" requirement, and a "security cam to a few viewers" requirement really narrows down your options to WebRTC Media Server.

Check out Kurento, Unreal Media Server, Ant Media Server.

  • Typically, the camera will stream video to the Media Server via RTSP. This is especially good since Opus is supported by RTSP/RTP; some IP cameras already support Opus audio (Axis brands).
  • Note that some media servers will transcode your video in order to stream to WebRTC viewers in web browsers, and this process takes a lot of CPU, thus, doesn't scale well. Try to choose a media server that uses original camera-encoded content, instead of transcoding.
user1390208
  • 1,866
  • 20
  • 20