1

I have an AXIS IP camera (M1054) which sends an H264/RTP stream via RTSP.

Unfortunately, they do not send SPS and PPS NALUs at all, they only transfer (fragmented) Codec slices.

I'm trying to decode that stream with the iOS VideoToolbox framework which needs the H264 SPS and PPS tuple to correctly setup the CMFormatDescription.

I wonder how I can synthesize the necessary parameter sets from looking at the actual H264 slices?

Update: I have captured an example session where mplayer manages to display the stream via Wireshark. The capture file is here and you can see the whole RTSP setup as well as a couple of seconds RTP.

DrMickeyLauer
  • 4,455
  • 3
  • 31
  • 67

1 Answers1

1

RTP consists of 3 sets of flows.

  1. RTP for the media
  2. RTSP for controlling the connection
  3. RTCP for the sender confirmation and timestamps.

Although the SPS/PPS is often in band inside the stream and is transported via RTP - it doesn't need to be there (and may be shouldn't be there). The SPS/PPS is transmitted as part of the setup process (RTSP). I usually recommend running http://www.live555.com/ in the debugger to learn about the details of the process - but http://www.live555.com/ is currently down.

In very rare circumstances you could recreate the SPS/PPS from a well known constrained H.264 stream. But in general you can't. So the SPS/PPS are metadata of the H.264 stream that is not redundantly stored anywhere else.

So if your familiarize yourself with the setup process - RTSP - it will be pretty obvious.

DrMickeyLauer
  • 4,455
  • 3
  • 31
  • 67
Markus Schumann
  • 7,636
  • 1
  • 21
  • 27
  • Is it the line `a=fmtp:96 packetization-mode=1; profile-level-id=420029; sprop-parameter-sets=Z0IAKeKQCgDLYC3AQEBpB4kRUA==,aM48gA==`? If so, how would I be able to encode this as SPS and PPS NALUs, which is what iOS/VideoToolbox needs? – DrMickeyLauer Mar 19 '19 at 16:36
  • Further inspection shows that `sprop-parameters-sets` is indeed the base64 encoded version of the SPS and PPS. Thanks a lot! – DrMickeyLauer Mar 19 '19 at 16:44
  • Any advice on what exactly is in there, how to decode or encode the sprop-parameter-sets? I cant find any practical info on how to get that if you're doing custom video encoding in code. – oarfish Sep 13 '21 at 14:40
  • sprop-parameter-sets are well defined in specifications and http://www.live555.com/ is a great sample implementation that contains encode and decode. I am not sure what other advice you are looking for. – Markus Schumann Sep 14 '21 at 14:34