2

From a Webrtc providing browser i receive an RTP stream which gets decrypted using janus gateway. Upon receiving only the video rtp packets get relayed to a local multicast group for testing purpose.

So, let's assume i receive vp8 encoded rtp packets on a udp port. I'm also able to request for a new keyframe at any time.

The problem pipeline:

gst-launch-1.0 -v -v -v -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 ! "application/x-rtp, payload=100, clock-rate=90000" ! rtpvp8depay ! webmmux streamable=true ! filesink location=/tmp/test.webm

produces the error

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "application/x-rtp\,\     payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:src: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:sink: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2933): gst_base_src_loop (): /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:00.039571113
Setting pipeline to PAUSED ...
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = video/webm
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

The Kurento project provides a gstreamer plugin called "vp8parse" which solves the issue:

gst-launch-1.0 -v -v -v -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 ! "application/x-rtp, payload=100, clock-rate=90000" ! rtpvp8depay ! vp8parse ! webmmux streamable=true ! filesink location=/tmp/test.webm
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:src: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/KmsVp8Parse:kmsvp8parse0.GstPad:src: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/KmsVp8Parse:kmsvp8parse0.GstPad:sink: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:sink: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
HERE THE PIPELINE BLOCKS UNTIL A KEYFRAME IS RECEIVED
/GstPipeline:pipeline0/KmsVp8Parse:kmsvp8parse0.GstPad:src: caps = "video/x-vp8\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)10/1"
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstMatroskamuxPad:video_0: caps = "video/x-vp8\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)10/1"
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = video/webm
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/webm
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = "video/webm\,\ streamheader\=\(buffer\)\<\ 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000502ad7b1830f42404d809f4753747265616d657220706c7567696e2076657273696f6e20312e342e31005741994753747265616d6572204d6174726f736b61206d7578657200446188062408b80e88c4001654ae6b010000000000003cae0100000000000033d7810183810173c588786b225315e5f279536e86566964656f00e00100000000000008b0820280ba8201e08686565f56503800\ \>"
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = "video/webm\,\ streamheader\=\(buffer\)\<\ 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000502ad7b1830f42404d809f4753747265616d657220706c7567696e2076657273696f6e20312e342e31005741994753747265616d6572204d6174726f736b61206d7578657200446188062408b80e88c4001654ae6b010000000000003cae0100000000000033d7810183810173c588786b225315e5f279536e86566964656f00e00100000000000008b0820280ba8201e08686565f56503800\ \>"

Having a look at the vp8parse source it seems that this plugin does nothing else than pipe the frames which rtpvp8depay provides untouched to its sink BUT also set's the src caps to the video width,height and framerate.

An alternative pipeline which works is:

gst-launch-1.0 -v -v -v -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 ! "application/x-rtp, payload=100, clock-rate=90000" ! rtpvp8depay ! vp8dec ! vp8enc ! webmmux streamable=true ! filesink location=/tmp/test.webm

but using vp8dec ! vp8enc obviously doesnt make much sense as i already receive a vp8 encoded stream.

Now my question is how can i solve this without reencoding the stream and without depending on vp8parse? If there is no alternative it seems i have to use it, but as this is currently a plugin which is not available via standard gstreamer plugin packages i would like to avoid this. Is it possible to force the caps to a specific width,height,framerate so webmmux wouldnt complain? Cause i think that's the reason why the very first pipeline is not-negotiated.

I tried using capsfilter like rtpvp8depay ! capsfilter caps="video/x-vp8,width=640,height=480,framerate=10/1" ! webmmux but it doesnt negotiate either.

mpromonet
  • 11,326
  • 43
  • 62
  • 91
John Doe
  • 2,746
  • 2
  • 35
  • 50
  • Why are you against using Kurento's plugin? The reason this happens is because Chrome changes the vp8 bitstream on the fly for bandwidth reasons and the way I found to correct it is both of the solutions you have already posed(using that element or re-encoding the stream). Part of the reason the plugin works but your forcing the caps does not, is that it reads the values from the keyframes and dynamically sets the caps so that the negotiation will work. Putting in a capsfilter does not do that. – Benjamin Trent Jan 09 '15 at 16:19
  • Im not sure i understand this correctly. Does this mean, that i could use capsfilter to set the framerate to a fixed value which may work for a few seconds and then break as chrome may change the stream? If yes, why does the pipeline break immediately during receiving of udp packets? It doesnt event wait for a keyframe. Your question: Mostly out of plain interest i try to understand how the codecs, browser, gstreamer, kurento plugin work together. I was hoping for a simple solution which helps understand non-kurento-plugin workflow better. – John Doe Jan 09 '15 at 18:34
  • Its not the framerate alone that is the issue, but also the picture resolution is changed on the fly. – Benjamin Trent Jan 10 '15 at 19:22

1 Answers1

0

It is bug 747208, already fixed upstream, but the version of Gstreamer in your system might be old (1.2.4 in Ubuntu 14.04) and still affected.

As a workaround in such old versions, if you know the frame size of the video you can use a capssetter element after the depayloader to manually set the caps that the depayloader misses:

gst-launch-1.0 -v \
  udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 \
  ! "application/x-rtp, payload=100, clock-rate=90000" \
  ! rtpvp8depay ! capsetter caps="video/x-vp8,width=640,height=480" \
  ! webmmux streamable=true ! filesink location=/tmp/test.webm
joanpau
  • 568
  • 5
  • 14