11

I couldn't find a solution for my previous question so I decided to try it step by step.

The thing that I wanna do now is to store the RTP/H264 stream as a file.

What I found so far is the below:

(firstly my RTP/H264 is FU-A which is the form of)

| RTP HEADER 12bytes long | FU INDICATOR 1byte | FU HEADER 1byte | FU payload |

As I understood RFC 6184 document, I'm starting a NAL with a packet which has '1' at the first bit of FU Header and appending following packets that set '0' at the first bit until the last packet which has '1' at the second bit of FU Header.

I think that's how to get a complete NAL before the FU-A packetization and also what I found is I need to put the 'starting bits' (0x00000001) at the front of each complete NAL.

But no luck so far. The below is part of log

 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]7C/1111100 [5]85/10000101 [6]B8/10111000 [7]40/1000000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]7C/1111100 [5]85/10000101 [6]B8/10111000 [7]40/1000000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]7C/1111100 [5]85/10000101 [6]B8/10111000 [7]40/1000000 ...
 ========= adding the next NAL as 716
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]7C/1111100 [5]85/10000101 [6]B8/10111000 [7]40/1000000 ...
 ========= a NAL is summed up as 4866
 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 139
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= a NAL is summed up as 7061
 ========= the new NAL is as 1377
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]41/1000001 [5]E4/11100100 [6]40/1000000 [7]1A/11010 ...
 ========= a NAL is summed up as 1369
 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E6/11100110 [7]60/1100000 ...
 ========= adding the next NAL as 94
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E6/11100110 [7]60/1100000 ...
 ========= a NAL is summed up as 1472
 ========= the new NAL is as 447
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]41/1000001 [5]E8/11101000 [6]80/10000000 [7]16/10110 ...
 ========= a NAL is summed up as 439
 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]EA/11101010 [7]A0/10100000  ...
 ========= adding the next NAL as 1174
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]EA/11101010 [7]A0/10100000 ...
 ========= a NAL is summed up as 2552
 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]EC/11101100 [7]C0/11000000 ...
 ========= adding the next NAL as 1364
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]EC/11101100 [7]C0/11000000 ...
 ========= a NAL is summed up as 2742
 ========= the new NAL is as 1400
 ...

my question is,

  1. if I can get the complete NALs from the fragmented packets by FU-A, how can I make it as a file which is able to run by VLC or other player?

  2. I'm still confused if I have to keep the FU indicator and FU header or not. Somebody said I need to take them only for the very first packet (starting with '1' at the FU header)

Any advice will be really appreciated.

Thanks.

Community
  • 1
  • 1
Jun
  • 181
  • 2
  • 3
  • 7
  • Hey, I am trying to do the exact same thing.. I read the RFC 6184 and tried to follow teh same steps and store it into a file.. But my file does not seem to play at all... Can you please post the solution you used? This question has over 7k views.. It will be helpful for all of us.. – Sandeep Nov 28 '16 at 20:14

2 Answers2

6

The FU, STAP, and MTAP NAL units are specific only to RTP packetization as they're designed to facilitate network transport. In other words, don't count on a decoder parsing them correctly. Ultimately, you need to reassemble NAL units as in the case of FU packets, or break them up in to multiple NAL units the case of STAP/MTAP.

Once you have a NAL unit (and this includes PPS, SPS, SEI, slice partitions, and all the other types in the 1-23 range), then you can write to disk along with the '0001' start codes per H.264 annex B.

Placing the H.264 annex B stream into a container such as MPEG-4 can be done with various command-line tools (I'm pretty sure ffmpeg can do it).

Jeff Wallace
  • 61
  • 1
  • 2
5
  1. One option is to mux the data into a file format such as mp4 or avi to be able to play it with VLC. AFAIR avi was a bad fit for H.264 (can't remember the reason off hand). There are free libraries such as libmp4, or on if you're on windows using DirectShow, Geraint's mp4mux.

Another option is to use ffmpeg to convert a .264 file into an mp4

ffmpeg -i test.264 test.mp4

This assumes that the .264 file contains NAL units separated by start codes.

  1. From RFC6184

The FU payload consists of fragments of the payload of the fragmented NAL unit so that if the fragmentation unit payloads of consecutive FUs are sequentially concatenated, the payload of the fragmented NAL unit can be reconstructed. The NAL unit type octet of the fragmented NAL unit is not included as such in the fragmentation unit payload, but rather the information of the NAL unit type octet of the fragmented NAL unit is conveyed in the F and NRI fields of the FU indicator octet of the fragmentation unit and in the type field of the FU header. An FU payload MAY have any number of octets and MAY be empty.

You would need to reconstruct the original NAL unit before writing it to the .264 file if you take the second approach outlined in 1.

Community
  • 1
  • 1
Ralf
  • 9,405
  • 2
  • 28
  • 46
  • Thanks again Ralf, btw I'm really wondering what I should do with the sps, pps. Currently the complete NAL that I'm reconstructing doesn't have sps, pps packets. I think I have to put them before the very first packet of the stream or need to have a AVCodecContext initialized with the sps, pps. How do you think?? – Jun Mar 09 '12 at 04:18
  • Ok, I made some progress. I put the sps, pps packets at the head of the stream and edited each reconstructed single nal's header from 2bytes (it still consisted of FU indicator, FU header) into 1byte as it was originally. now elecard stream analyzer reads the file but still not readable for vlc or any other player but if I converted this into mp4 by 'ffmpeg -i src.mp4 con.mp4', the con.mp4 can be played by general video players. – Jun Mar 10 '12 at 06:48
  • But I need to decode the file by ffmpeg api in my own player, so I have to make it run without the conversion. It should be decoded by ffmpeg api avcodec_decode_video2().. any advice???? – Jun Mar 10 '12 at 06:52
  • just pass the reconstructed NAL units to avcodec_decode_video2() I can send a sample code if you need it. – arash kordi Oct 03 '12 at 08:17
  • i visited this post after a long time since I wrote it. Thanks for all of your help. I solved it at that time and actually the final step I needed to do was to rename the output file's extension. players were not that smart to recognize the file is flawless if it has a different extension. – Jun Dec 07 '12 at 04:31
  • @Jun, pleased you got this working. Do you know anything about trying to simultaneously write an H.264 stream to mp4 file whilst reading an earlier segment? – noelicus Feb 14 '13 at 09:28