RealityKit – Loading Reality Composer scenes with SwiftUI

Question

I'm trying to load different models on face using SwiftUI, RealityKit and ARKit.

struct AugmentedRealityView: UIViewRepresentable {

    @Binding var modelName: String

    func makeUIView(context: Context) -> ARView {
    
        let arView = ARView(frame: .zero)
    
        let configuration = ARFaceTrackingConfiguration()

        arView.session.run(configuration, options: [.removeExistingAnchors, 
                                                    .resetTracking])
    
        loadModel(name: modelName, arView: arView)
    
        return arView
    
    }

    func updateUIView(_ uiView: ARView, context: Context) { }

    private func loadModel(name: String, arView: ARView) {

        var cancellable: AnyCancellable? = nil
    
        cancellable = ModelEntity.loadAsync(named: name).sink(
                 receiveCompletion: { loadCompletion in
            
            if case let .failure(error) = loadCompletion {
                print("Unable to load model: \(error.localizedDescription)")
            }                
            cancellable?.cancel()
        },
        receiveValue: { model in
            
            let faceAnchor = AnchorEntity(.face)
            arView.scene.addAnchor(faceAnchor)
            
            faceAnchor.addChild(model)
            
            model.scale = [1, 1, 1]
        })
    }
}

This is how I load them but when the camera view opens and loads one model then the other models won't be loaded. Can someone help me out?

Read this post to find out how to do it: https://stackoverflow.com/questions/59618102/multi-face-detection-in-realitykit/59809624#59809624 — Andy Jazz, Aug 18 '20 at 09:21

score 0 · Answer 1 · edited May 25 '21 at 17:31

When the value of your Binding changes, SwiftUI is calling your updateUIView(_:,context:) implementation, which does noting.

Additionally, you are not storing the AnyCancellable. When the token returned by sink gets deallocated the request will be cancelled. That could result in unexpected failures when trying to load lager models.

To fix both of these issue, use a Coordinator.

import UIKit
import RealityKit
import SwiftUI
import Combine
import ARKit

struct AugmentedRealityView: UIViewRepresentable {
    class Coordinator {
        private var token: AnyCancellable?
        private var currentModelName: String?
        
        fileprivate func loadModel(_ name: String, into arView: ARView) {
            // Only load model if the name is different from the previous one
            guard name != currentModelName else {
                return
            }
            currentModelName = name
            
            // This is optional
            // When the token gets overwritten
            // the request gets cancelled
            // automatically
            token?.cancel()
            
            token = ModelEntity.loadAsync(named: name).sink(
                receiveCompletion: { loadCompletion in
                    
                    if case let .failure(error) = loadCompletion {
                        print("Unable to load model: \(error.localizedDescription)")
                    }
                },
                receiveValue: { model in
                    
                    let faceAnchor = AnchorEntity(.camera)
                    arView.scene.addAnchor(faceAnchor)
                    
                    faceAnchor.addChild(model)
                    
                    model.scale = [1, 1, 1]
                })
            }
        
        fileprivate func cancelRequest() {
            token?.cancel()
        }
    }
    
    @Binding var modelName: String
    
    func makeCoordinator() -> Coordinator {
        Coordinator()
    }
    
    static func dismantleUIView(_ uiView: ARView, coordinator: Coordinator) {
        coordinator.cancelRequest()
    }
    
    func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)
        
        let configuration = ARFaceTrackingConfiguration()
        
        arView.session.run(configuration, options: [.removeExistingAnchors,
                                                    .resetTracking])
        
        context.coordinator.loadModel(modelName, into: arView)
        
        return arView
        
    }
    
    func updateUIView(_ uiView: ARView, context: Context) {
        context.coordinator.loadModel(modelName, into: uiView)
    }
}

We create a nested Coordinator class that holds the AnyCancellable token and move the loadModel function into the Coordinator. Other than a SwiftUI View, the Coordinator is a class that lives while your view is visible (always a remember that SwiftUI might create and destroy your View at will, its lifecycle is not related to the actual "view" that is shown on screen).

In out loadModel class we double check that the value of our Binding actually changed so that we don't cancel an ongoing request for the same model when SwiftUI updates our View, e.g. because of a change in the environment.

Then we implement the makeCoordinator function to construct one of our Coordinator objects. Both in makeUIView and in updateUIView we call the loadModel function on our Coordinator.

The dimantleUIView method is optional. When the Coordinator gets deconstructed our token gets released as well, which will trigger Combine into canceling ongoing requests.

RealityKit – Loading Reality Composer scenes with SwiftUI

1 Answers1