CMSampleBufferRef pool to write H.264 AVCC stream

Question

I'm using AVAssetWriter/AVAssetWriterInput to write H.264 raw data to an MP4 file. As I'm receiving the data from a remote server, I use the following CoreMedia APIs to get a sample buffer (CMSampleBufferRef) containing the H.264 data in AVCC format that is in turned appended to an MP4 file by sending to an AVAssetWriterInput the message (BOOL)appendSampleBuffer:(CMSampleBufferRef)sampleBuffer:

CMBlockBufferCreateWithMemoryBlock to create a memory block

CMBlockBufferReplaceDataBytes to write the H.264 in AVCC format to the memory block

CMSampleBufferCreate to create a sample buffer with the memory block and a format descriptor containing the H.264 "extradata"

Everything works as expected, the only problem with this approach is that I'm periodically calling the above APIs and what I would really like is instead to be able to reuse the resources allocated - in particular CMSampleBufferRef and CMBlockBufferRef. Basically, I would like to have a pool of CMSampleBuffer's and be able to update its memory content and format descriptor as I'm receiving new H.264 data from the remote server.

I know that exists AVAssetWriterInputPixelBufferAdaptorthat gives access to a CVPixelBufferPool but, I can't use it in my case because as far as I know, to properly instantiate a pixel buffer adaptor, at minimum I need to be able to pass the video frame dimensions which I would' know until I parse the stream. Further, I don't know how to write the H.264 "extradata" with a CVPixelBuffer. So, I'm thinking that I need to stick with CMSampleBuffer. Unfortunately, it seems that CoreMedia APIs don't offer the possibility to update the memory block nor the format descriptor of a sample buffer once created (as far as I can tell, I only have access to immutable references of those objects). Thus, the best I can do so far is to reuse the memory block CMBlockBufferRef but I'm still recreating the sample buffer. My code is below. Hopefully someone here will have some ideas on how to implement a pool of CMSampleBuffer's or perhaps a more efficient way to write H.264 AVCC stream to MP4?

- (CMSampleBufferRef)sampleBufferWithData:(NSData*)data formatDescriptor:(CMFormatDescriptionRef)formatDescription
{
    OSStatus result;

    CMSampleBufferRef sampleBuffer = NULL;

    // _blockBuffer is a CMBlockBufferRef instance variable
    if (!_blockBuffer)
    {
        size_t blockLength = MAX_LENGTH;
        result = CMBlockBufferCreateWithMemoryBlock(kCFAllocatorDefault,
                                                    NULL,
                                                    blockLength,
                                                    kCFAllocatorDefault,
                                                    NULL,
                                                    0,
                                                    blockLength,
                                                    kCMBlockBufferAssureMemoryNowFlag,
                                                    &_blockBuffer);

        // check error
    }

    result = CMBlockBufferReplaceDataBytes([data bytes], _blockBuffer, 0, [data length]);

    // check error

    const size_t sampleSizes = [data length];

    CMSampleTimingInfo timing = [self sampleTimingInfo];

    result = CMSampleBufferCreate(kCFAllocatorDefault,
                                  _blockBuffer,
                                  YES,
                                  NULL,
                                  NULL,
                                  formatDescription,
                                  1,
                                  1,
                                  &timing,
                                  1,
                                  &sampleSizes,
                                  &sampleBuffer);

    // check error

    return sampleBuffer;
}

score 1 · Answer 1 · edited Jul 18 '18 at 06:25

If you are receiving raw H.264 data, then there is not much do to and no need to deal with CoreMedia at all.

Buffer all VCL NAL units until you get SPS/PPS NAL units. Create the extradata from them, then just append all buffered and new VCL NAL units to the file. In case you are received the NAL units in Annex B format you need to convert them to AVCC format (basically replacing the start code with a length code)

You only need to work with 'CMSampleBuffer' if you want to decode uncompressed pictures or if you want to decode compressed pictures. As you are already working with a raw H.264 stream and just want to write it into an MP4 file, just do so. No need to touch CoreMedia at all here.

Regarding CoreMedia: you wrap your video information in a CMBlockBuffer. This buffers together with a CMVideoFormatDescriptor (generated from SPS/PPS) plus CMTime make up a CMSampleBuffer. And multiple CMSampleBuffers make up a 'CMSampleBufferPool'.

'CVPixelBuffer' and 'CVPixelBufferPool' are not involved. These are either the input or output of a 'VTCompressionSession' or "VTDecompressionSession' when dealing with encoding/decoding h.264 video.

As said in your case, no need to touch any of the core framworks at all as you are just creating a file.

An overview about Annex B and AVCC stream format can be found here: Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream

CMSampleBufferRef pool to write H.264 AVCC stream

1 Answers1