Wrong Vision Framework Landmarks Coordinates

Question

I'm trying to capture face landmarks with Vision Framework to show them on screen, but the eyes always appear a little higher than expected, like the Tim Cook image below.

Here is my capturing code:

guard let pixelBuffer = CMSampleBufferGetImageBuffer(cmSampleBuffer) else { return }
var requests: [VNRequest] = []
    
let requestLandmarks = VNDetectFaceLandmarksRequest { request, _ in
    guard let results = request.results as? [VNFaceObservation],
          let firstFace = results.first else { return }

    completion(self.drawFaceWithLandmarks(face: firstFace))
}

requests.append(requestLandmarks)
    
let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: .leftMirrored)
do {
    try handler.perform(requests)
} catch {
    print(error)
}

Here is how I'm converting the Vision coordinates

let transform = CGAffineTransform(scaleX: 1, y: -1).translatedBy(x: 0, y: -UIScreen.main.bounds.height)
let translate = CGAffineTransform.identity.scaledBy(x: UIScreen.main.bounds.width, y: UIScreen.main.bounds.height)
let facebounds = face.boundingBox.applying(translate).applying(transform)

let eyePathPoints = eye.normalizedPoints
    .map({ eyePoint in
        CGPoint(
            x: eyePoint.x * facebounds.width + facebounds.origin.x,
            y: (1-eyePoint.y) * facebounds.height + facebounds.origin.y)
        })

Has anyone ever faced a similar problem?

Wrong Vision Framework Landmarks Coordinates

0 Answers0