Save what user see in AVCaptureVideoPreviewLayer

Question

I'm trying to develop an app with custom camera where the user can add filters or sticker (like in TextCamera app) and share in social feed. But I found my first problem.

I show the preview to the user with AVCaptureVideoPreviewLayer, take the photo and pass it to another view controller in a UiImageView but the second picture is bigger than first one.

I tried to resize the picture with this function:

    func resize(image: UIImage) -> UIImage {
       let size = image.size
       let newWidth = CGFloat(size.width)
       let newHeight = CGFloat(size.height - blackBottomTab.bounds.size.height)

       let newSize = CGSizeMake(newWidth,newHeight)
       let rect = CGRectMake(0, 0, newSize.width, newSize.height)

       UIGraphicsBeginImageContextWithOptions(newSize, false, 1.0)
       image.drawInRect(rect)
       let newImage = UIGraphicsGetImageFromCurrentImageContext()
       UIGraphicsEndImageContext()

       return newImage
}

In this function I subtract the height of the black view (under the button) from the image height. But the result that I have is different (see the photo attached).

This is my preview with a black view under the button

This is the photo taken larger than preview one

I also tried to use Aspect Fit in Storyboard Image View of the second View Controller but the result is the same.

Where is my error? Thank you to everyone that help me!

Answer 1

I think that the AVCaptureVideoPreviewLayer frame is the same as the screen's frame (UIScreen.mainScreen().bounds), and you added the "Shoot Photo" black view on top of it. Instead, you should change the frame of the AVCaptureVideoPreviewLayer.

Your case (what I think):

Assuming that the green rectangle is the AVCaptureVideoPreviewLayer frame and the red one is the black view frame. So, it covers (on top) of the green rectangle.

Make them look like this:

Hope that helped.

Answer 2

I had to solve a similar problem. As the question notes, there does not appear to be an easy way to detect the size of the video preview.

My solution is hinted at at the end of the answer to https://stackoverflow.com/a/57996039/10449843 which explains in detail how I take the snapshot and create the combined snapshot with the sticker.

Here is an elaboration of that hint.

While I use AVCapturePhotoCaptureDelegate to take the snapshot, I also use AVCaptureVideoDataOutputSampleBufferDelegate to sample the buffer once when the preview is first shown to detect the proportions of the snapshot.

            // The preview layer is displayed with aspect fill
            let previewLayer = AVCaptureVideoPreviewLayer(session: session)
            previewLayer.videoGravity = .resizeAspect
            previewLayer.frame = self.cameraView.bounds

Detect the size of the preview:

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
        // Only need to do this once
        guard self.sampleVideoSize == true else {
            return
        }

        guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
            return
        }

        // Sample the size of the preview layer and then store it
        DispatchQueue.main.async {

            let width = CGFloat(CVPixelBufferGetWidth(pixelBuffer))
            let height = CGFloat(CVPixelBufferGetHeight(pixelBuffer))

            // Store the video size in a local variable
            // We don't care which side is which, we just need the
            // picture ratio to decide how to align it on different
            // screens.
            self.videoSize = CGSize.init(width: width, height: height)

            // Now we can set up filters and stickers etc
            ...

            // And we don't need to sample the size again
            self.sampleVideoSize = false
        }
        return
    }