ARKit: How can I add a UIView to ARKit Scene?

Question

I am working on an AR project using ARKit.

I want to add a UIView to ARKit Scene. When I tap on an object, I want to get information as a "pop-up" next to the object. This information is in a UIView.

Is it possible to add this UIView to ARKit Scene? I set up this UIView as a scene and what can I do then? Can I give it a node and then add it to the ARKit Scene? If so, how it works?

Or is there another way?

Thank you!

EDIT: Code of my SecondViewController

class InformationViewController: UIViewController {

    @IBOutlet weak var secondView: UIView!

    override func viewDidLoad() {
        super.viewDidLoad()
        self.view = secondView
    }
}

EDIT 2: Code in firstViewController

guard let secondViewController = storyboard?.instantiateViewController(withIdentifier: "SecondViewController") as? SecondViewController else {
    print ("No secondController")
    return
}

let plane = SCNPlane(width: CGFloat(0.1), height: CGFloat(0.1))       
plane.firstMaterial?.diffuse.contents = secondViewController.view

let node = SCNNode(geometry: plane)

I only get a white screen of a plane, not the view.

Answer 1

The simplest (although undocumented) way to achieve that is to set a UIView backed by a view controller as diffuse contents of a material on a SCNPlane (or any other geometry really, but it works best with planes for obvious reasons).

let plane = SCNPlane()
plane.firstMaterial?.diffuse.contents = someViewController.view
let planeNode = SCNNode(geometry: plane)

You will have to persist the view controller somewhere otherwise it's going to be released and plane will not be visible. Using just a UIView without any UIViewController will throw an error.

The best thing about it is that it keeps all of the gestures and practically works just as a simple view. For example, if you use UITableViewController's view you will be able to scroll it right inside a scene.

I haven't tested it on iOS 10 and lower, but it's been working on iOS 11 so far. Works both in plain SceneKit scenes and with ARKit.

Answer 2

I cannot provide you code now but this is how to do it.

1. Create a SCNPlane.
2. Create your UIView with all elements you need.
3. Create image context from UIView.
4. Use this image as material for SCNPlane.

Or even easier make SKScene with label and add it as material for SCNPlane.

Example: https://stackoverflow.com/a/74380559/294884

Answer 3

To place text in a label in the world you draw it into an image and then attach that image to a SCNNode.

For example:

let text = "Hello, Stack Overflow."
let font = UIFont(name: "Arial", size: CGFloat(size))
let width = 128
let height = 128

let fontAttrs: [NSAttributedStringKey: Any] = 
[NSAttributedStringKey.font: font as UIFont]

let stringSize = self.text.size(withAttributes: fontAttrs)
let rect = CGRect(x: CGFloat((width / 2.0) - (stringSize.width/2.0)),
                  y: CGFloat((height / 2.0) - (stringSize.height/2.0)),
                  width: CGFloat(stringSize.width),
                  height: CGFloat(stringSize.height))

let renderer = UIGraphicsImageRenderer(size: CGSize(width: CGFloat(width), height: CGFloat(height)))
let image = renderer.image { context in

    let color = UIColor.blue.withAlphaComponent(CGFloat(0.5))

    color.setFill()
    context.fill(rect)

    text.draw(with: rect, options: .usesLineFragmentOrigin, attributes: fontAttrs, context: nil)
}

let plane = SCNPlane(width: CGFloat(0.1), height: CGFloat(0.1))
plane.firstMaterial?.diffuse.contents = image

let node = SCNNode(geometry: plane)

EDIT: I added these lines:

let color = UIColor.blue.withAlphaComponent(CGFloat(0.5))

color.setFill()
context.fill(rect)

This lets you set the background color and the opacity. There are other ways of doing this - which also let you draw complex shapes - but this is the easiest for basic color.

EDIT 2: Added reference to stringSize and rect