Vision Framework: While tracking the user's face how can I have an image (like a hat) move in sync with the face?

Question

I am trying to do the following:

1. Start front facing camera
2. Detect a person’s face
3. Place 3D/image overlay over their face (a hat, for example)

Using the example project from apple (here) the first two steps work fine. However now I want to complete the third step and add an UIImage as a subview and have it move in sync with the face so that the hat (for example) stays in place on top of the user's head while they move the camera.

I am able to add the image and have it move with the face somewhat editing this code from the example project:

fileprivate func addIndicators(to faceRectanglePath: CGMutablePath, faceLandmarksPath: CGMutablePath, for faceObservation: VNFaceObservation) {
    let displaySize = self.captureDeviceResolution

    let faceBounds = VNImageRectForNormalizedRect(faceObservation.boundingBox, Int(displaySize.width), Int(displaySize.height))
    faceRectanglePath.addRect(faceBounds)

    if let landmarks = faceObservation.landmarks {
        // Landmarks are relative to -- and normalized within --- face bounds
        let affineTransform = CGAffineTransform(translationX: faceBounds.origin.x, y: faceBounds.origin.y)
            .scaledBy(x: faceBounds.size.width, y: faceBounds.size.height)

        // Treat eyebrows and lines as open-ended regions when drawing paths.
        let openLandmarkRegions: [VNFaceLandmarkRegion2D?] = [
            landmarks.leftEyebrow,
            landmarks.rightEyebrow,
            landmarks.faceContour,
            landmarks.noseCrest,
            landmarks.medianLine
        ]
        for openLandmarkRegion in openLandmarkRegions where openLandmarkRegion != nil {
            self.addPoints(in: openLandmarkRegion!, to: faceLandmarksPath, applying: affineTransform, closingWhenComplete: false)
        }

        // Draw eyes, lips, and nose as closed regions.
        let closedLandmarkRegions: [VNFaceLandmarkRegion2D?] = [
            landmarks.leftEye,
            landmarks.rightEye,
            landmarks.outerLips,
            landmarks.innerLips,
            landmarks.nose
        ]
        for closedLandmarkRegion in closedLandmarkRegions where closedLandmarkRegion != nil {
            self.addPoints(in: closedLandmarkRegion!, to: faceLandmarksPath, applying: affineTransform, closingWhenComplete: true)
        }



        self.imageView.frame = CGRect(x: faceBounds.origin.x, y: faceBounds.origin.y, width: 300, height: 165)
        self.previewView?.addSubview(self.imageView)

    }
}

But I am not sure what X and Y values to use for the imageView.frame because using faceBounds.origin the result is not very consistent.

Here is the link to the modified sample project if you want to try it: https://files.fm/f/zqzrd6gf

Thank in advance.