CMBlockBuffer ownership in CMSampleBuffer

Question

I'm writing code to decompress a native annex-B H.264 stream, and I'm going through the process of parsing the stream, creating a CMVideoFormatDescription from the SPS/PPS NALUs, and wrapping the other NALUs I extract from the stream in CMSampleBuffers.

I'm suffering from a mental block on how to handle the CMBlockBuffer and CMSampleBuffer memory for the decoder. I believe my issue is more of a lack of thorough understanding of how CF handles memory than anything else, so my question is really more about that, but I'm hoping the context is helpful.

If I create a CMBlockBuffer like this:

CMBlockBufferRef blockBuffer;

OSStatus status = CMBlockBufferCreateWithMemoryBlock(NULL,
                                                     memoryBlock,                       
                                                     blockBufferLength,
                                                     kCFAllocatorNull,
                                                     NULL,
                                                     0, 
                                                     blockBufferLength,          
                                                     kCMBlockBufferAlwaysCopyDataFlag | kCMBlockBufferAssureMemoryNowFlag,
                                                     &blockBuffer);

and add it to a CMSampleBuffer like this:

CMSampleBufferRef sampleBuffer;

status = CMSampleBufferCreate(kCFAllocatorDefault,
                              blockBuffer,
                              true,
                              NULL,
                              NULL,
                              formatDescription,
                              1,
                              0,
                              NULL,
                              1,
                              &sampleSize,
                              &sampleBuffer);

How should I handle the block buffer? Does the SampleBuffer retain the memory of the block buffer, or do I need to do something to make sure it is not deallocated?

Also, pertaining to the asynchronous decode process, is there a sensible way to know when the decoder is done with the CMSampleBuffer so I can dispose of it?

My intuition tells me the CMSampleBuffer would retain the CMBlockBuffer, and the VTDecodeSession would retain the CMSampleBuffer until it's done decoding, but this is undocumented territory I'm wandering in so looking for some direction. The results I'm getting imply my intuition might be wrong, so I need rule out memory management as an issue to keep my sanity...

score 5 · Accepted Answer · answered Dec 01 '16 at 20:33

5

CMSampleBuffers and CMBlockBuffers - the objects themselves - follow typical CF Retain/Release semantics. You should hold a retain so long as you need these objects, and assume that the interfaces that accept them do the same. This means you are free to release the CMBlockBuffer as soon as you hand it to a CMSampleBuffer, and you're free to release the CMSampleBuffer after you pass it into the rendering chain.

The memory pointed to by a CMBlockBuffer created with CMBlockBufferCreateWithMemoryBlock() follows slightly different rules. First, that method does not copy the data pointed to by memoryBlock; it uses that pointer directly. This means the BlockBuffer needs to have some knowledge of how that memory should be managed. This is handled by either the fourth or fifth arguments to CMBlockBufferCreateWithMemoryBlock(): if either are non-kCFAllocatorNull/NULL, the BlockBuffer will call the deallocator of one of those when it's done with the memory. This is typically done in the BlockBuffer's Finalize(). If they're both kCFAllocatorNull/NULL (which you have in your code snippet), the BlockBuffer will just drop the pointer on the floor when it's done with memory.

This means that if you create a CMBlockBuffer using CMBlockBufferCreateWithMemoryBlock() and intend to release your retain on that BlockBuffer after passing it down the rendering pipeline, you should use non-NULL arguments for the allocator/deallocators so the memory can be reclaimed later. The implementations of these allocators/deallocators, of course, is dependent on the origin of the memoryBlock.

answered Dec 01 '16 at 20:33

DSaracino

241
2
6

Thanks, this is helpful. So if I'm using malloc to allocate the CMBlockBuffer, what would the right allocator value be? – Jim B. Dec 01 '16 at 22:13
One additional note, I did get this to work, but I discovered that another data structure handed to VTDecompressionSessionCreate, the CMVideoFormatDescription, appears not to be copied by the session, and I needed to make sure I held onto that block as well or I get a crash. The crash for these conditions appears in assembly in the CFEqual routine, and if you look one level up you see it was trying to read the CMVideoFormatDescription at that time. – Jim B. Dec 01 '16 at 22:16
1

The documentation for kCFAllocatorMalloc says it uses malloc, realloc, and free. If I were to go that route, though, I'd probably use the same allocator allocate the memory directly rather than assume my call to malloc() matched the allocator's call to free(). Seems safer and more explicit/easier to understand, even though the documentation claims they're the same. – DSaracino Dec 02 '16 at 02:57
Regarding the crash: Hmm... This sounds like the kind of thing that a ton of people would hit if the VTDecompressionSession didn't hold onto the FormatDescription as long as it needed to. It's hard to know without without looking at the client code. Are you sure you don't have an extra release? File a radar? – DSaracino Dec 02 '16 at 03:04
I'll try to isolate the crash condition better. Haven't filed a radar since I worked at Apple 23 years ago. I can still do that? – Jim B. Dec 02 '16 at 14:18
Thanks, amazing answer. It made me realize in my code I was using the wrong blockAllocator for the CMBlockBufferCreateWithMemoryBlock! – bojan Dec 18 '20 at 12:24

CMBlockBuffer ownership in CMSampleBuffer

1 Answers1