2

I am using Janus Gateway to stream a video taken from the video file and encoded by ffmpeg. The goal of the project is much more complicated but i tried to reduce it to the minimum possible to help troubleshooting.

encoding is done this way:

ffmpeg -re -thread_queue_size 4 -stream_loop -1 -i \
/root/syncTest.mkv -c:v libvpx -vf scale="-1:360" -minrate 400k \
-maxrate 600k -b:v 500k -an -deadline realtime -f rtp \ 
rtp://127.0.0.1:5004?pkt_size=1200 -vn -ar 48000 -ac 2 -c:a libopus \
 -f rtp rtp://127.0.0.1:5002?pkt_size=1200

I am using Streaming plugin and stream appears to be configured with no tricks:

stream-1 :
{
  type = "rtp";
  id = "1";
  description = "stream-1";
  audio = "yes";
  audioport = "5002";
  audiopt = "111";
  audiortpmap = "opus/48000/2";
  video = "yes";
  videoport = "5004";
  videopt = "100";
  videortpmap = "VP8/90000";
  data = "no";
  secret = "adminpwd";
};

Here are the links to my streaming plugin and Janus configs - i tried to play a bit with nack queue length and dtls timeout parameters but to no avail, they don't change anything.

https://codeda.com/data/janus.jcfg.txt https://codeda.com/data/janus.plugin.streaming.jcfg.txt

Resulting video (screen capture of a WebRTC player page) is here: https://codeda.com/data/janusSyncIssue.mp4 - as you can see, there is a considerable unsync between video and audio.

As i see it, the problem is that the video stream and audio stream are sent separately and either do not contain timecodes, or they are for some reason ignored. Obviously encoding video and encoding audio takes different time, more for video, so that results in a delayed video.

Am i right suspecting this? If so, how can i fix this? I imagine there is some ffmpeg setting or Janus streaming plugin setting for that.

Ah yes, processor is almost free, 80-85% free, on that server. Not a processor issue for sure.

Alexander Novikov
  • 503
  • 1
  • 3
  • 14
  • So far no one was able to help. Sad. There's always an unsync - unstable and depending on playing device, browser, etc., plus variable over time, it shifts. It's also not depending on codec - exhibits about same behavior on H264 too. – Alexander Novikov Sep 15 '19 at 13:56
  • Problem is fixed but this isn't an answer since i still don't know how to do it with ffmpeg, while i'm certain it must work. With gstreamer it worked out of the box. – Alexander Novikov Sep 16 '19 at 17:15
  • ```gst-launch-1.0 -v rtpbin name=rtpbin filesrc location=out.mkv ! matroskademux name=demux demux.audio_0 ! rtpopuspay ! rtpbin.send_rtp_sink_0 demux.video_0 ! rtpvp8pay ! rtpbin.send_rtp_sink_1 rtpbin.send_rtp_src_0 ! udpsink host=janus6.codeda.com port=5002 sync=true async=false rtpbin.send_rtcp_src_0 ! udpsink host=janus6.codeda.com port=5003 sync=false async=false rtpbin.send_rtp_src_1 ! udpsink host=janus6.codeda.com port=5004 sync=true async=false rtpbin.send_rtcp_src_1 ! udpsink host=janus6.codeda.com port=5005 sync=false async=false``` – Alexander Novikov Sep 16 '19 at 17:15

0 Answers0