2

I am currently using Vision with ARKit to find any faces in the frame. The code I'm using to do this is below:

func runFaceDetection() {
        let pixelBuffer : CVPixelBuffer? = (sceneView.session.currentFrame?.capturedImage)
        if pixelBuffer == nil { return }
        let ciImage = CIImage(cvPixelBuffer: pixelBuffer!)
        let imageRequestHandler = VNImageRequestHandler(ciImage: ciImage, options: [:])

        do {
            try imageRequestHandler.perform(self.visionRequests)
        } catch {
            print(error)
        }
}

I am using the following code to attempt to map these returned rectangles to a given face:

func faceRectanglesCompletionHandler(request: VNRequest, error: Error?) {
        if error != nil {
            print("error: " + (error?.localizedDescription)!)
            return
        }

        guard let faces = request.results else {
            print("No faces found in frame")
            return
        }

        DispatchQueue.main.async {
            for faceRect in self.faceRects {
                faceRect.removeFromSuperview()
            }

            self.faceRects = []

        }

        for face in faces {
            let observation = face as! VNFaceObservation
            print(observation.boundingBox)

            let newWidth = bounds.width * observation.boundingBox.width
            let newHeight = bounds.height * observation.boundingBox.height
            let newX = bounds.width * observation.boundingBox.origin.x
            let newY = bounds.height * observation.boundingBox.origin.y
            let newBoundingBox = CGRect(x: newX, y: newY, width: newWidth, height: newHeight)

            DispatchQueue.main.async {
                let faceView = UIView.init(frame: newBoundingBox)
                faceView.backgroundColor = .red
                faceView.alpha = 0.5
                self.faceRects.append(faceView)
            }
        }

        DispatchQueue.main.async {
            for faceRect in self.faceRects {
                self.view.addSubview(faceRect)
            }
        }
    }

The problem is that this is not correctly mapping the face rectangles to points on the screen. The bounds property of my viewController is set as bounds = sceneView.bounds, where sceneView is my AR scene. When the face rectangles appear, which they often don't they will appear as elongated rectangles with too much height and not enough width. What am I doing wrong in my mapping code? An image of the problem is below. enter image description here

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
Alex Wulff
  • 2,039
  • 3
  • 18
  • 29

0 Answers0