5

I am new to gstreamer and I want to stream a mp4 video which is having audio and video both from my Host(Ubuntu PC) to target board. I have successfully streamed only video. But streaming both(audio and video) is not happening.

Pipeline to stream only video is as below

Sender:

gst-launch-1.0 -v filesrc location = video.mp4 ! decodebin ! x264enc ! rtph264pay ! udpsink host=192.168.2.136 port=5000

receiver:

gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96, ssrc=3394826012, timestamp-offset=2215812541, seqnum-offset=46353" ! rtph264depay ! avdec_h264 ! videoconvert ! ximagesink sync=false

Hence, can see video on target.

Did streaming of only audio from mp4 file.

Pipeline to stream only video is as below

Sender:

gst-launch-1.0 -v filesrc location=video.mp4 ! qtdemux name=demuxer demuxer. ! queue ! rtpmp4apay pt=97 ! udpsink host=192.168.2.69 port=5000

Receiver:

gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp, media=audio, clock-rate=44100, encoding-name=MP4A-LATM, cpresent=0, config=40002420, payload=97, ssrc=3386264266, timestamp-offset=2822202855, seqnum-offset=17719" ! rtpmp4adepay ! decodebin ! alsasink &

This too runs successfully.

But when I tried below pipeline to stream both audio/video in sync:

Sender:

gst-launch-1.0 -v filesrc location=sample.mp4 ! qtdemux name=demux demux. ! queue ! rtph264pay pt=96 ! udpsink host=192.168.3.101 port=5000 demux. ! queue ! rtpmp4apay pt=97 ! udpsink host=192.168.3.101 port=5001

Receiver:

gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96, ssrc=2226494377, timestamp-offset=3242004369, seqnum-offset=17021" ! rtph264depay ! decodebin ! ximagesink udpsrc port=5001 caps="application/x-rtp, media=audio, clock-rate=44100, encoding-name=MP4A-LATM, cpresent=0, config=40002420, payload=97, ssrc=1499706502, timestamp-offset=190741668, seqnum-offset=24774" ! rtpmp4adepay ! decodebin ! alsasink

But: get an output as below:

ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin1/GstFaad:faad0: Could not decode stream.

fails. Please can someone suggest me a pipeline which can stream audio/video both on target device in sync.

Thanks in advance.

Sandeep Kumar
  • 336
  • 1
  • 3
  • 9

3 Answers3

2

There seem to be issues with AAC in RTP as well as other RTP payloaders in gstreamer.

As mentioned in the other answer, it is anyway the best strategy to not demux and split video and audio if it is desired to have a synchronized playback on the receiver side. Additionally, a container format that is streamable can also improve overall functionality.

My proposal is therefore to remux the data streams into MPEG2-TS and transport that via RTP. This has several advantages, like good re-synchronization after packet loss and that A/V always stays sync.

Example

This works for me with gstreamer 1.2.4:

Sender

gst-launch-1.0 -v filesrc location=/foo/bar.mp4 ! \
  qtdemux name=demux demux. ! h264parse config-interval=10 ! queue ! mux. \
  mpegtsmux name=mux ! rtpmp2tpay ! udpsink host=10.0.0.1 port=5003 \
  demux. ! aacparse ! queue ! mux.

Receiver (file)

gst-launch-1.0 -v udpsrc port=5003 caps="application/x-rtp" ! \
  rtpmp2tdepay ! tsparse !  filesink location=/foo/bar.ts

Receiver (output)

This command is based on the commands in the question, could not test it with ALSA:

gst-launch-1.0 -v udpsrc port=5003 caps="application/x-rtp" ! \
  rtpmp2tdepay ! tsparse ! tsdemux name=demux \
  demux. ! queue ! audioconvert ! alsasink \
  demux. ! queue ! xvimagesink
Roy D'atze
  • 146
  • 6
  • Receiver side for directly playing video and audio wont work for me. – Sandeep Kumar Mar 08 '16 at 11:08
  • Something like an error message or program output would be a little bit more helpful! – Roy D'atze Mar 09 '16 at 10:07
  • Hi! Your example works for me. But only whrn I launch Server and Client commands on local machine. If I try to broadcast video to another PC I have next erorr: `ERROR: from element /GstUDPSrc:source: Could not get/set settings from/on resource./GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstUDPSrc:source: bind failed: Error binding to address: The requested address is not valid in its. Whats happened with streaming? – JDo Aug 30 '18 at 12:05
  • JDo, this sounds like an unrelated issue and you should create a new post with it, including gstreamer output etc. – Roy D'atze Aug 30 '18 at 16:04
1

There is no easy way to stream in sync with gstreamer if you demux the video and audio stream on the sender side. Timestamp information is kept in the container (MP4 in this case), which is lost when demuxing.

Try to send the MP4 over to the receiver, and then demux there. A basic example may be found here: Streaming Mp4 video through RTP protocol using Gstreamer in Ubuntu

Community
  • 1
  • 1
Roy D'atze
  • 146
  • 6
  • thanks for reply but I am able to receive video but link provided by you is useless for me as i need to stream audio too. If you have some idea that how can I stream audio from mp4 then please suggest it to me. – Sandeep Kumar Mar 07 '16 at 12:39
  • Since it shows a decoding error: did you cross-check, e.g. with ffprobe, that the MP4 file contains AAC audio data? – Roy D'atze Mar 07 '16 at 12:56
  • Yes on applying ffprob output is Stream #0.1(eng): Audio: aac, 44100 Hz, stereo, s16, 93 kb/s Metadata: creation_time : 2014-10-28 18:06:24 Can you please provide a pipeline that can stream only audio from mp4. So I can verify that audio is getting streamed properly as I am not able to do it. – Sandeep Kumar Mar 07 '16 at 13:34
  • I can get it working without the UDP part: .... ! qtdemux name=demux demux. ! queue ! fakesink demux. ! queue ! rtpmp4apay pt=97 ! rtpmp4adepay ! decodebin ! audioconvert .... But the decoding behind UDP is really buggy. I can investigate later. – Roy D'atze Mar 07 '16 at 17:02
0

Sender

gst-launch-1.0 filesrc location= ~/1.mp4 ! qtdemux name=d ! queue ! h264parse ! rtph264pay config-interval=10 pt=96 ! udpsink host=10.24.217.30 port=5000 d. ! queue ! aacparse ! rtpmp4apay pt=97 ! udpsink host=10.24.217.30 port=5001 -v

Receiver

gst-launch-1.0 udpsrc port=5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H264" ! rtph264depay ! identity silent=0 ! avdec_h264 ! videoconvert ! ximagesink sync=0 async=0 udpsrc port=5001 caps="application/x-rtp, media=audio, clock-rate=44100, encoding-name=MP4A-LATM, cpresent=0, config=40002420, payload=97, ssrc=1499706502, timestamp-offset=190741668, seqnum-offset=24774" ! rtpmp4adepay ! avdec_aac ! identity silent=0 ! alsasink sync=0 async=0

This works properly.

Note: Starting receiver early before starting sender gives error at the receiver end when sender is started. ** (gst-launch-1.0:26959): CRITICAL **: 15:15:51.191: gst_audio_decoder_set_output_format: assertion 'GST_AUDIO_INFO_IS_VALID (info)' failed ERROR: from element /GstPipeline:pipeline0/avdec_aac:avdec_aac0: GStreamer error: negotiation problem. Additional debug info: gstavauddec.c(449): gst_ffmpegauddec_negotiate (): /GstPipeline:pipeline0/avdec_aac:avdec_aac0: Could not set caps for libav decoder (aac), not fixed?

This could be a bug in GST in caps negotiations. WAR is to start receiver after sender.

Viranjan Pagar
  • 21
  • 3