CIImage CVPixelBuffer is nil after scaling down

Question

I'm using the AVVideoComposition API to get CIImages from a local video, and after scaling down the CIImage I'm getting nil when trying to get the CVPixelBuffer.
Before scaling down the source frame, I'm getting the original frame CVPixelBuffer.
Is there any reason the buffer is nil after scaling down?

Sample:

   AVVideoComposition(asset: asset) { [weak self] request in
        let source = request.sourceImage
        let pixelBuffer = source.pixelBuffer // return value
        let scaledDown = source.transformed(by: .init(scaleX: 0.5, y: 0.5))
        let scaledPixelBuffer // return nil
   })

Can you try getting the pixel buffer using this? https://developer.apple.com/documentation/coreimage/cicontext/1437853-render — Asteroid, May 29 '22 at 16:51
Hey @Asteroid. Yes, I can! But it's less efficient, I'm trying to understand if I can just access it directly after scaling so I won't do another pass — Roi Mulia, May 29 '22 at 16:54

score 1 · Accepted Answer · answered May 29 '22 at 18:14

I think the last line in your sample is incomplete. Did you mean let scaledPixelBuffer = scaledDown.pixelBuffer? If so, then yes, this won't work. The reason is that the pixelBuffer property is only available if the CIImage was created directly from a CVPixelBuffer. From the docs:

If this image was create using the init(cvPixelBuffer:) initializer, this property’s value is the CVPixelBuffer object that provides the image’s underlying image data. […] Otherwise, this property’s value is nil.

The CIImage that is passed to the composition block was created from a pixel buffer provided by AVFoundation. But when you apply a filter or transform to it, you need to render the resulting image into a pixel buffer explicitly using a CIContext, otherwise you won't get a result.

If you want to change the size of the video frames the composition is using, you can use a AVMutableVideoComposition instead and set its renderSize to your desired size after it is initialized:

let composition = AVMutableVideoComposition(asset: asset) { … }
composition.renderSize = CGSize(width: 1280, height: 720)

As always Frank, best in the business. Thanks, buddy! – Roi Mulia May 29 '22 at 18:29 — Roi Mulia, May 29 '22 at 18:29

CIImage CVPixelBuffer is nil after scaling down

1 Answers1