0

I've been trying to have a live RTSP streaming of video and audio over a single stream.

What I did was very simillar to their example (creation of ServerMediaSession and adding two SubSessions to it, one for video and one for audio), the only change I've made is that I've made a new bytestream source called TcpSource, which is very simillar to their ByteStreamFileSource, only instead of "fread()", I call recv() in doReadFromFile method (video and audio have different sockets, off course).

The result is that if I have each subsession seperately, it works fine. However, when I try to stream both video and audio via TCP, it causes either major packet losses, or having only one stream working properly while the other stuck in the middle (e.g video is freezing while audio keeps playing fine).

Can you please advise? Does it has something to do with sending\receiving timeouts? Thanks in advance.

DanielY
  • 1,141
  • 30
  • 58

1 Answers1

1

The live555 event loop is single-threaded and is sensitive to any blocking or heavy processing.

The first thing to check is that you are not using blocking I/O. If you are, you are interfering with all the things the live555 framework takes care of, e.g. any timers, RTCP reporting, socket reads and writes, etc.

The second thing is that you need to check that you are not blocking in your doReadFromFile. System calls like sleep are not recommended. Instead use the live555 task scheduler mechanism.

I have written similar code i.e. reading audio/video from tcp sockets, parsing the packets, and then re-broadcasting the media via live555. I handled all my network I/O in a separate thread and passed parsed media samples into the live555 thread via a shared queue mechanism. This has worked well so far, but there are lots of other ways you could approach this e.g. using the live555 thread for I/O like you are doing should be fine as long as you don't interfere with the live555 event loop.

Ralf
  • 9,405
  • 2
  • 28
  • 46
  • Hi @Ralf, thanks for the reply, and sorry for the delay in comment...Can you please elaborate what do you mean by "blocking i\o"? Is recv() from socket considered as blocking i\o? debug prints? – DanielY Nov 01 '15 at 05:55
  • @DanielY What OS are you using? recv() could be blocking or non-blocking depending on how you've configured the socket. If you haven't explicitly made the socket non-blocking, it's likely blocking. This means that the call does not return until there is something to read on the socket which when using live555 causes all sorts of issues. Say you're blocking on reading video, in the mean time x audio packets could already have been read, RTCP needed to be sent, etc. – Ralf Nov 01 '15 at 07:30
  • I'm currently using win7 64-bit of my PC. I've configured the sockets to be non-blocking and even though I get many errors of "non-blocking operation could not be completed immediately", the video\audio does not stuck (only not synched to each other, but that's another issue). Any idea why I get the above error when using recv()? – DanielY Nov 01 '15 at 08:32
  • That's the correct behaviour for non-blocking sockets: it means that if there is nothing to be read the recv() operatio will return that error code: don't do anything in your function and let the live555 event loop continue. If there is something to be read, then process the data as you were doing previously. – Ralf Nov 01 '15 at 08:58
  • Ok i'll try that, thanks a lot. Is there any way to reach you if I need anything else? – DanielY Nov 01 '15 at 09:18
  • Hi @Ralf, it seems that my problem wasn't just the blocking recv(), but also bad presentation times...I still can't figure out how to fix it...how did you do it in your application? – DanielY Nov 04 '15 at 07:26
  • I convert DirectShow timestamps to the equivalent presentation times used by live555. It's very specific to our system though. – Ralf Nov 04 '15 at 10:17