Is there a way to pass a ModelEntity to ARQuickLook preview controller?

Question

I'm working on a project where I have to download a USDZ file from a URL, preconfigured with white materials, then customize it in runtime and finally view it in AR with ARQuickLook.

At the moment, I thought the best way was to download the asset using the ModelEntity download method, change its properties and then show it with the ARQuickLook preview. Currently, I am completely stuck in the last step where I am looking for the way to pass the modified model entity to the ARQuickLook preview controller, but it only accepts a URL and no other data types.

A simple code example below:

var modelURL: URL?

override func viewDidLoad() {
    super.viewDidLoad()

    self.downloadUSDZ()
}

@IBAction func arQuickLookButtonPressed(_ sender: Any) {
            
    guard modelURL != nil else { return }
    let previewController = QLPreviewController()
    previewController.dataSource = self
            
    present(previewController, animated: true, completion: nil)
}

func downloadUSDZ() {
    
    modelURL = URL(string: "https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/toy_drummer.usdz")!
    
    guard let entity = try? ModelEntity.loadModel(contentsOf: modelURL!) else {
        print("Entity download failed")
        return
    }
    
    for child in entity.children {
                    
        var newMaterial = SimpleMaterial()
        newMaterial.color.tint = UIColor.cyan
        
        child.model?.materials = [newMaterial]
    }                
}

func numberOfPreviewItems(in controller: QLPreviewController) -> Int { return 1 }
   
func previewController(_ controller: QLPreviewController, previewItemAt index: Int) -> QLPreviewItem {

    let previewItem = ARQuickLookPreviewItem(fileAt: modelURL!) //<---- HERE I NEED TO DISPLAY THE MODIFIED MODEL ENTITY
    previewItem.canonicalWebPageURL = URL(string: "https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/")
    previewItem.allowsContentScaling = false
    return previewItem
}

Can anyone give me some advice on how to proceed? Other ways to reach the goal are also accepted.

Have you solved this issue? I'm looking for an answer for this as well. What i have found so far is, ARQuickLook is not intended for things like this. It is in-fact designed to display a quick preview of an existing model with some built in options to manipulate the object (transform/scale etc). Only way i could archive above is by using an ARView. — Ryan Aluvihare, Feb 17 '23 at 17:04
I simply used the SceneKit API instead of the ModelEntity. In particular, I downloaded the model in this way `let scene = try SCNScene(url: modelSourceURL)`, then I changed the materials of the model and finally I saved it locally using the method `scene.write(to: modelDestinationURL)`. This way I could pass the local modelDestinationURL to the ARQuickLookPreviewItem. Important, create a `SCNView(frame: .zero)` and assign the SCNScene to it before writing the file, otherwise the changes on the model will not be saved (this drove me crazy for a few days). Hope this will help you. — zico, Feb 18 '23 at 18:26
That's a great tip. Thanks! Btw, Have you found a way to identify which material is which instead of looping through them and assigning the new material? So we can assign different material to each surface accurately — Ryan Aluvihare, Feb 19 '23 at 07:52

score 0 · Answer 1 · answered Feb 17 '23 at 17:25

I'm not sure if this is doable with ARQuickLook. But we can use either SceneKit or RealityKit ARView and modify the ModelEntity at runtime. You could do something like this, Using ARView in RealityKit:

@IBOutlet var arView: ARView!


override func viewDidLoad() {
    super.viewDidLoad()

    let modelURL = URL(string: "https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/toy_drummer.usdz")!

    guard let entity = try? ModelEntity.loadModel(contentsOf: modelURL!) else {
        print("Entity download failed")
        return
    }
    

    for child in entity.children {
                    
        var newMaterial = SimpleMaterial()
        newMaterial.color.tint = UIColor.cyan
        
        child.model?.materials = [newMaterial]
    }  

    let anchor = AnchorEntity(plane: .horizontal)
    anchor.addChild(entity)

    arView.scene.addAnchor(anchor)
      
}

Please keep in mind that you will have to manually add the transform/scale actions that you get automatically with ARQuickLook.

Is there a way to pass a ModelEntity to ARQuickLook preview controller?

1 Answers1