0

I'm trying to figure out how RTSP works, when the handshake is complete. Does the client have to query the server for each new piece of data? Or the server sends data all the time and doesn't care how the client receives the data?

The reason why I ask this is my Gstreamer pipeline receiving RTSP stream from IP camera:

uridecodebin -> nvstreammux -> queue1 -> nvinfer -> queue2 -> fakesink

The IP camera has 30FPS but the nvinfer element can process only 10FPS. So I assumed that pending frames are stored in queue1 element waiting to be processed. However the current number of buffers of the queue1 is 1 all the time.

So one possible answer is that frames or packets are queued in one of the elements of uridecodebin element, but I did not find any such element there. Next it can mean that the IP camera was informed by uridecodebin to decrease FPS. Or if the uridecodebin element has to ask for each new piece of data, it just means that it asks for new data only when all frames are processed in the pipeline.

Do you have any idea?

harry_tums
  • 91
  • 7

1 Answers1

1

First it is worth mentioning that RTSP is a control protocol - the actual video media is usually sent over RTP protocol in most 'RTSP' video streaming cases.

RTSP is a streaming control protocol, to control streaming servers (whoami's remote control analogy here is nice: https://stackoverflow.com/a/43045354/334402) It defines how to package data to stream it and how both ends of the connection should behave to support the protocol.

So RTSP does not actually transport the media data itself - as mentioned above it is usually RTP (Real Time Transport) that does this.

Back to your question - RTCP and RTP can be transferred over UDP or TCP in IP Networks. In the former case, a socket is opened a stream established and the client can sit back and wait for the packets to arrive. In the latter case, the client requests each packet as you outline.

In practice you will likely find TCP is more commonly used, even if it is not necessarily the best match for streamed video, because it is more suited to traversing the many firewalls and NAT's on the internet.

In fact it gets more complicated sometimes with the RTSP data interleaved with RTP and RTCP (Real Tine Control Protocol that collects statistics about the flow) packets over TCP - you can see most detail by looking at the IEFT RFC: https://datatracker.ietf.org/doc/html/rfc2326

Mick
  • 24,231
  • 1
  • 54
  • 120