12

I have the FFMPEG streaming baseline h264 video, which I have to encapsulate in RTP and send to SIP phones for their decoding. I am using Linphone with the h264 plugin for Windows and Mirial for the decoding progress. However, sometimes I get a huge frame size (3Kb ~ 9Kb) from the FFMPEG, which obviously doesn't fit in the MTU.

If I send these frames "as is" and trusting IP fragmentation feature, some phones are able to play it well enough, but others choke and can't decode the stream. I think this is because the stream is not compliant with the RFC 3984 that specifies that packets that don't fit in the MTU have to be separated into different NALUs and mark the end of a Frame with the Mark feature of RTP.

How do I know where I can "cut" the I or P frame? I noticed that fragmented h264 packets (the ones without the Mark label) sometimes finish in 0xF8 but couldn't quite get a pattern and in the RFC 3984 which describes how to send these packets over RTP doesn't specify how to do it.

UPDATE: Does anyone know how to tell the X264 library how to generate NALUs of a Max Size? that way i should be able to avoid this problem. Thanks everyone

Pierluigi Cifani
  • 135
  • 1
  • 2
  • 9

2 Answers2

15

As an author to RFC 3984bis (to be RFC 6184), it details exactly how to convert H.264 NALs into RFC 3984 packets. There are 3 modes: 0 (single-NAL), 1 (allows for fragmenting and combining NALs), and 2 (lets you fragment, combine, and interleave the transmission order to change how a burst loss will affect a stream, among other things). See SDP packetization-mode. Only mode 0 is required.

Mode 0 (Single-NAL) requires you either use UDP fragmentation (discouraged) or tell the encoder don't generate NALs larger than MTU-X. You should be able to tell the encoder this.

Mode 1 lets you fragment. See the RFC for how you set up an FU-A packet. The fragmentation info is on the front. You can also use STAPs to aggregate small NALs like SPS and PPS packets sent before IDRs (normally). Each packet requires normal RTP headers with incremented sequence numbers (but the same timestamp).

Mark on the last RTP packet of a frame (not of a fragment or NAL) is expected but you shouldn't count on it.

jesup
  • 6,765
  • 27
  • 32
  • @jesusp Thank you for your response! however, I am using X264 as the encoder layer and I am not aware of a way of telling the encoder to generate NALUs of 1500 bytes tops. Googling around got me here: [link](http://comments.gmane.org/gmane.comp.video.x264.devel/5460). However, it seems it hasn't been merged into the library. Does anyone know a way of doing it using X264? Otherwise I will have to implement the packetization-mode=1. Thanks everyone – Pierluigi Cifani Apr 11 '11 at 15:02
  • @jesup Can you explain why UDP fragmentation is bad ? As I understand it, UDP fragmentation (or better, IP fragmentation) should lead to less network traffic since you don't have to encode a RTP (respectively UDP) header for each packet (so you gain from 12 to 20 bytes per packet). – xryl669 Jun 05 '15 at 13:46
  • There's much verbiage on this... Some routers don't support fragmentation at all. Various OS's and other intermediary boxes may need to reassemble the fragmented packet, and all such have limits on the reassembled size - typically far below the max UDP size of 65535 bytes (often around 4K). There's no standard max size. fragmentation leaves you more vulnerable to packet loss (though if the data is unusable in single packets that may not help much). Perhaps this should be revisited, but the lack of a useful standard size is a big problem. – jesup Jun 11 '15 at 12:07
6

In x264, I believe the int i_slice_max_size in x264_param_t can be used to control the size. Have a look in x264.h I can't remember where I read this, but the post said this structure member can be used to control the NAL size, but I haven't tried it myself.

int i_slice_max_size; /* Max size per slice in bytes; includes estimated NAL overhead. */

EDIT: I found the source

http://mailman.videolan.org/pipermail/x264-devel/2011-February/008263.html

ianhobo
  • 200
  • 2
  • 8
  • Thanks! I'm getting into it this afternoon, but I already checked and it's not going to be a problem to apply these changes. Thanks a lot for your help! – Pierluigi Cifani Apr 26 '11 at 09:08