ARKit 1.5: Extract the recognized image

Question

So, my goal is:

• Find known image
• Extract it from the sceneView (e.g take a snapshot)
• Perform further processing

It was quite easy to complete 1st step using ARReferenceImage :

guard let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil) else { return }

let configuration = ARWorldTrackingConfiguration()

configuration.detectionImages = referenceImages

But now I can't figure out how to extract the image form the SceneView. I have the plane node added to the imageAnchor:

    func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
    DispatchQueue.main.async { [unowned self] in
        guard let imageAnchor = anchor as? ARImageAnchor
            else { return }

        let planeNode = self.getPlaneNode(withReferenceImage: imageAnchor.referenceImage)
        planeNode.opacity = 0.5
        planeNode.eulerAngles.x = -.pi / 2
        node.addChildNode(planeNode)
    }
}

And the result:

So that, I need the planeNode's projection on the screen to get node's 2D screen coordinates and then cut the image. I found this method: renderer.projectPoint(node.position) but it didn't help me. It can even return negative values when the whole picture is on the screen..

Am I doing it the correct way? Any help would be very appreciated .

Answer 1

In my case I solved it.

    let transform = node.simdConvertTransform(node.simdTransform, to: node.parent!)

    let x = transform.columns.3.x
    let y = transform.columns.3.y
    let z = transform.columns.3.z

    let position = SCNVector3(x, y, z)
    let projection = renderer.projectPoint(position)

    let screenPoint =  CGPoint(x: CGFloat(projection.x), y: CGFloat( projection.y))

For more info

ARKit 2 - Crop Recognizing Images

iOS 8 Core Image saving a section of an Image via Swift

https://nacho4d-nacho4d.blogspot.com/2012/03/coreimage-and-uikit-coordinates.html