0

I am working on analyzing H264 video data being streamed over a network. Right now, I am able to successfully extract and analyze the raw H264 for UDP. This process is going to be ALOT harder for the TCP/RTSP because of fragmentation and multiplexing.

Is the video compression / encoding any different on the TCP/RTSP multiplexed stream compared to the UDP stream?

user3335040
  • 649
  • 1
  • 7
  • 17
  • 1
    The video _encoding_ should be identical - what'll change will be having to decode TCP streams on the fly. account for segment offsets, flags, retransmits, etc. That ain't easy - I've tried...! – Alnitak Aug 04 '14 at 21:30
  • Thanks. So there probobly are not any tools / sourceware which extracts H264 from the TCP/RTSP fragmentation, right? Second question, can I ask questions about finding tools for this on stack? – user3335040 Aug 04 '14 at 21:32
  • are you sniffing packets off the wire, or reading them after they've been processed by your local TCP/IP stack? – Alnitak Aug 04 '14 at 22:04
  • Offline Pcap files from wireshark. – user3335040 Aug 04 '14 at 22:06
  • Right, in which case my comments about TCP stream reassembly stand :( Good luck! – Alnitak Aug 04 '14 at 22:07

1 Answers1

3

It's only slightly harder as you typically have to demultiplex the audio and video, as well as the RTCP reports on the TCP connection. Fragmentation is not an issue.

Is the video compression / encoding any different on the TCP/RTSP multiplexed stream compared to the UDP stream?

No differences at all. The multiplexing of RTP/RTCP packets is defined in RFC2326.

As far as tools go, you can use openRTSP from http://www.live555.com which handles the transport for you (RTP over RTSP via the -t command line argument) and writes the frames to file.

With reference to Ainitak's comment, it's not that complex: there's a 4 byte header, '$' followed by the channel id, followed by the 2 byte length of the following RTP/RTCP packet. It's not too tricky to parse this.

Community
  • 1
  • 1
Ralf
  • 9,405
  • 2
  • 28
  • 46
  • I was referring to reassembling TCP streams themselves, not the TCP/RTSP payloads. – Alnitak Aug 04 '14 at 21:43
  • I'm not sure I understand what you're referring to: the only difference between UDP and TCP transport is that you need to extract the RTP or RTCP packets from the tcp stream. This can be done by parsing the 4-byte header. Once you've extracted the packet, there's no difference with respect to the RTP packet? – Ralf Aug 04 '14 at 21:49
  • _once you've extracted the packet_ - that's the hard bit - on first reading I believed that the OP was sniffing packets over the wire, at which point you have to handle IP frames and TCP segments yourself. OTOH, if this is just a decoding exercise on a socket that already went through an O/S IP stack then it's pretty easy per the RFC you quoted. – Alnitak Aug 04 '14 at 22:05
  • Ok, understand :-) . I understood that the OP was handling the TCP connection. – Ralf Aug 04 '14 at 22:44
  • Extactly so any tools that you can suggest for taking in PCAP files and dealing with these protocols would be amazing! We are looking at VideoSnarf, live555 (Does not handle Pcap files though), and libpcap. – user3335040 Aug 04 '14 at 23:36