Face detection swift vision kit

Question

I am trying Vision kit for iOS 11. I can use Vision and I can find boundbox values face. But I don't know how can I draw a rectangle using this points. I hope so my question is clear.

What have you done till now and what do you want to do from now? Please show by code. — OOPer, Jul 25 '17 at 10:02

score 6 · Accepted Answer · answered Jul 25 '17 at 10:36

Hope you were able to use VNDetectFaceRectanglesRequest and able to detect faces. To show rectangle boxes there are lots of ways to achieve it. But simplest one would be using CAShapeLayer to draw layer on top your image for each face you detected.

Consider you have VNDetectFaceRectanglesRequest like below

let request = VNDetectFaceRectanglesRequest { [unowned self] request, error in
            if let error = error {
                // somthing is not working as expected
            }
            else {
                //  we got some face detected
                self.handleFaces(with: request)
            }
        }
        let handler = VNImageRequestHandler(ciImage: ciImage, options: [:])
        do {
            try handler.perform([request])
        }
        catch {
           // catch exception if any
        }

You can implement a simple method called handleFace for each face detected and use VNFaceObservation property to draw a CAShapeLayer.

func handleFaces(with request: VNRequest) {
        imageView.layer.sublayers?.forEach { layer in
            layer.removeFromSuperlayer()
        }
        guard let observations = request.results as? [VNFaceObservation] else {
            return
        }
        observations.forEach { observation in
            let boundingBox = observation.boundingBox
            let size = CGSize(width: boundingBox.width * imageView.bounds.width,
                              height: boundingBox.height * imageView.bounds.height)
            let origin = CGPoint(x: boundingBox.minX * imageView.bounds.width,
                                 y: (1 - observation.boundingBox.minY) * imageView.bounds.height - size.height)

            let layer = CAShapeLayer()
            layer.frame = CGRect(origin: origin, size: size)
            layer.borderColor = UIColor.red.cgColor
            layer.borderWidth = 2

            imageView.layer.addSublayer(layer)
        }
    }

More info can be found here in Github repo iOS-11-by-Examples

I accepted your answer. But my problem why x = boundingBox.minX * imageView.bounds.width and y = (1 - observation.boundingBox.minY) * imageView.bounds.height - size.height) — Mayday, Jul 25 '17 at 10:42
it is because we have to scale and transform according to our original image view else it will look weird in different places. — Bluewings, Jul 25 '17 at 10:47
For anyone else who is confused why you need to do `1 - observation.boundingBox.minY` it's because the coordinates of the bounding box are normalized to the dimensions of the processed image, with the origin at the image's lower-left corner. See: [https://stackoverflow.com/a/45317950/6942666](this) — Eric Wiener, Dec 10 '19 at 18:22

score 1 · Answer 2 · answered Jul 28 '17 at 11:25

Here is easy and simple way to draw boxes.

let faceRequest = VNDetectFaceRectanglesRequest(completionHandler:self.faceDetection)

func faceDetection (request: VNRequest, error: Error?) {
        guard let observations = request.results as? [VNFaceObservation]
            else { print("unexpected result type from VNFaceObservation")
                return }
        guard observations.first != nil else {
            return
        }
        // Show the pre-processed image
        DispatchQueue.main.async {
            self.resultImageView.subviews.forEach({ (subview) in
                subview.removeFromSuperview()
            })
            for face in observations
            {
                let view = self.CreateBoxView(withColor: UIColor.red)
                view.frame = self.transformRect(fromRect: face.boundingBox, toViewRect: self.analyzedImageView)
                self.analyzedImageView.image = self.originalImageView.image
                self.resultImageView.addSubview(view)                
        }
    }
}

 //MARK - Instance Methods
func boxView(withColor : UIColor) -> UIView {
    let view = UIView()
    view.layer.borderColor = withColor.cgColor
    view.layer.borderWidth = 2.0
    view.backgroundColor = UIColor.clear
    return view
}


//Convert Vision Frame to UIKit Frame
func transformRect(fromRect: CGRect , toViewRect :UIView) -> CGRect {

    var toRect = CGRect()
    toRect.size.width = fromRect.size.width * toViewRect.frame.size.width
    toRect.size.height = fromRect.size.height * toViewRect.frame.size.height
    toRect.origin.y =  (toViewRect.frame.height) - (toViewRect.frame.height * fromRect.origin.y )
    toRect.origin.y  = toRect.origin.y -  toRect.size.height
    toRect.origin.x =  fromRect.origin.x * toViewRect.frame.size.width

    return toRect
}

is there a way to do this with text detection only showing boxes around text inside a certain area, such as a rectangle in the centre of the screen and not on text outside of the box? — Tony Merritt, Sep 08 '17 at 13:37
With vision text detection that puts a rectangle around text that it finds. I am looking to have a mask of some sort so only text in the mask area shows te boxes around it and not outside the mask. — Tony Merritt, Sep 11 '17 at 20:02
I am not sure but, I think you can pass image part(masked) to detect the text after cropping it, so that you can detect the text within the rectangle. — Rajender Kumar, Sep 12 '17 at 07:19

Face detection swift vision kit

2 Answers2