Crop CMSampleBufferRef

Question

I am trying to crop image in CMSampleBufferRef to a specific size. I am making 5 steps - 1. Getting PixelBuffer from SampleBuffer 2. Converting PixelBuffer to CIImage 3. Cropping CIImage 4. Rendering CIImage back to PixelBuffer 5. Attaching PixelBuffer to SampleBuffer. So far I am having trouble with step 4 - rendering image back to PixelBuffer (can not check beyond this point) nothing is rendered to buffer (I check it using same CIImage imageWithCVPixelBuffer and get NULL as a return). Would greatly appreciate any tip or help.

CGRect cropRect = CGRectMake(0, 0, 640, 480);

CIImage *ciImage = [CIImage imageWithCVPixelBuffer:(CVPixelBufferRef)CMSampleBufferGetImageBuffer(sampleBuffer)]; //options: [NSDictionary dictionaryWithObjectsAndKeys:[NSNull null], kCIImageColorSpace, nil]];
ciImage = [ciImage imageByCroppingToRect:cropRect];    

CVPixelBufferRef pixelBuffer;
CVPixelBufferCreate(kCFAllocatorSystemDefault, 640, 480, kCVPixelFormatType_32BGRA, NULL, &pixelBuffer);

CVPixelBufferLockBaseAddress( pixelBuffer, 0 );

CIContext * ciContext = [CIContext contextWithOptions: nil];
[ciContext render:ciImage toCVPixelBuffer:pixelBuffer];
CVPixelBufferUnlockBaseAddress( pixelBuffer, 0 );

CMSampleTimingInfo sampleTime = {
    .duration = CMSampleBufferGetDuration(sampleBuffer),
    .presentationTimeStamp = CMSampleBufferGetPresentationTimeStamp(sampleBuffer),
    .decodeTimeStamp = CMSampleBufferGetDecodeTimeStamp(sampleBuffer)
};

CMVideoFormatDescriptionRef videoInfo = NULL;
CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, pixelBuffer, &videoInfo);

CMSampleBufferRef oBuf;
CMSampleBufferCreateForImageBuffer(kCFAllocatorDefault, pixelBuffer, true, NULL, NULL, videoInfo, &sampleTime, &oBuf);

What is the problem you are having? I don't see a question here. — Aurelius, Mar 28 '13 at 23:28
ciContext is not rendering anything in buffer. I am checking it with the same ciimage imageWithCVPixelBuffer and get NULL — Laz, Mar 28 '13 at 23:30
Hi. I am also stuck some where like you. I am capturing video using AVFoundation. I need to implement zoom in teh Avcapture delegate "didOutputSampleBuffer". Please tell me how to scale and crop a CMSampleBufferRef/CVImageBufferRef. — iOS_DEV, Jul 22 '14 at 11:20
If you do not need to control position, feed following settings to AVAssetWriterInput and it will autocrop NSDictionary* settings = [NSDictionary dictionaryWithObjectsAndKeys: AVVideoCodecH264, AVVideoCodecKey, [NSNumber numberWithInt: YOUR_SIZE_HERE], AVVideoWidthKey, [NSNumber numberWithInt: YOUR_SIZE_HERE], AVVideoHeightKey, [NSString stringWithString:AVVideoScalingModeResizeAspectFill], AVVideoScalingModeKey, // cropping nil]; — Laz, Jul 22 '14 at 17:34
If you need to control the position of the crop then I guess the only way (as of iOS6) would be to crop CVImageBufferRef using CIImage methods. Not sure if something changed in iOS7, haven't had time to go through updates yet — Laz, Jul 22 '14 at 17:37
Hi @iOS_DEV did you find the solution of your problem? I need exactly this. — benhi, Feb 03 '16 at 13:22
I posted my answer below, or you're asking about something else? — Laz, Feb 04 '16 at 07:27

Laz · Accepted Answer · 2013-03-29T23:04:23.783

So I found the reason why I was having problem. Apparently I have to add kCVPixelBufferIOSurfacePropertiesKey key when initializing pixel buffer, by creating dictionary and passing it to initializer. Here's a code if anyone stumbles upon this problem:

CFDictionaryRef empty; // empty value for attr value.
CFMutableDictionaryRef attrs;
empty = CFDictionaryCreate(kCFAllocatorDefault, // our empty IOSurface properties dictionary
                           NULL,
                           NULL,
                           0,
                           &kCFTypeDictionaryKeyCallBacks,
                           &kCFTypeDictionaryValueCallBacks);
attrs = CFDictionaryCreateMutable(kCFAllocatorDefault,
                                  1,
                                  &kCFTypeDictionaryKeyCallBacks,
                                  &kCFTypeDictionaryValueCallBacks);

CFDictionarySetValue(attrs,
                     kCVPixelBufferIOSurfacePropertiesKey,
                     empty);

Then you just have to pass this dictionary as an argument to CVPixelBufferCreate

I found this solution here: http://allmybrain.com/2011/12/08/rendering-to-a-texture-with-ios-5-texture-cache-api/

Crop CMSampleBufferRef

1 Answers1