0

I am trying to depacketize a raw RTP H265 stream and reconstruct it so that it can be read by a decoder. I have been able to extract individual and fragmented units from the RTP buffers by identifying the NALs and FU details. However, I couldn't find precise details on handling the NALs to be put in front of the buffer for Fragmented units.

Here is what I do:

  1. When the payload header type is 49, I know it is a Fragmented unit.

  2. I then look for the third byte whose trailing 6 bytes give you the FU type/ NAL type and the first two bits tells if its a start or an end slice of the frame. Here is the code I have for reconstructing the NAL:

if(type == 49){

    unsigned char fu_header = (unsigned char) *(rtpPayloadPos+2);
    unsigned char fu_head_se = (fu_header & 0xC0) >> 6;
    nal_type = fu_header & 0x3F;
 
    switch(fu_head_se){

    case 0x02:         //nal start
    {
        payloadType = RTP_PAYLOAD_FU_START_H265_NAL;
        *h265DataPos = rtpPlayloadPos-3;
        *(*h265DataPos) = 0x00;
        *((*h265DataPos)+1) = 0x00;
        *((*h265DataPos)+2) = 0x00;
        *((*h265DataPos)+3) = 0x01;
        *((*h265DataPos)+4) = (fu_header << 1);
        *((*h265DataPos)+5) = 0x01;
        h265Len = len - headLen + 3; 
    }
    break;

    case 0x00:     //nal middle
    {
        *h265DataPos = rtpPlayloadPos+2;
        h265Len = len - headLen - 2;
        payloadType = RTP_PAYLOAD_FU_MIDDLE_H265_NAL;
    }
    break;
 
    case 0x01:     //nal end
    {
        *h265DataPos = rtpPlayloadPos+2;
        h265Len = len - headLen - 2;
        payloadType = RTP_PAYLOAD_FU_END_H265_NAL;
    }
    break;
        
    default:
        printf("Unknown fu head\n");
        return -1;

}

However, I am not sure if the way I am reconstructing the NAL is right. Since the decoder complains alot and the video is almost green and jittery. Can someone tell me how to reconstruct the FU frame properly ?

I have been referring the ITU H265 document and this link: https://datatracker.ietf.org/doc/html/draft-ietf-payload-rtp-h265-15.

Community
  • 1
  • 1
Old Markus
  • 836
  • 7
  • 12

1 Answers1

1

The RFC specification rfc 7798 has all the details this.

Simple steps to follow:

  1. Remove the 12 bytes RTP header and 3 bytes of FU header.
  2. Combine all the FU packets till the end packet is found/received to from a Video Encoded frame.
  3. The Combined all packets (if multiple packets form a frame) to form Video Frame, it can be feed to the decoder.
Community
  • 1
  • 1
mail2subhajit
  • 1,106
  • 5
  • 16
  • Thanks for the input. I am past that stage. I have combined all the FU packets and have written them into an H265 file. But when I view this file on ffplay it says this: "The cu_qp_delta -57 is outside the valid range". This looks like a reference to the QP range in the PPS parameter. However if I save a file directly from the stream using ffplay and compare my file with the one recorded using ffplay, their VPS, SPS and PPS look the same. But for some reason, my video doesn't play properly using the same ffplay. This makes me think I have not processed the FU info properly. – Old Markus Dec 17 '19 at 18:10
  • Make sure you Encoder output and Decoder input dumps are the same. you can dump them to file and do a binary compare - verify the same. – mail2subhajit Dec 18 '19 at 08:27
  • I fixed the issue. The issue was that, in intermediate and final slices of the Fragmented units, I was supposed to drop the first three bytes and then append them. However, I was dropping the first 2 bytes and then appending them. Its weird because the ffplay kept complaining about cu_qp_delta which put me offtrack. Thanks for the help @mail2subhajit – Old Markus Dec 18 '19 at 13:40