How can I make shadows in RealityKit for USDZ files?

Question

I use the code from this Medium story, but I didn't use RealityComposer. I added a simple sphere and plane on the anchor. But the result is weird, the sphere casts three lights but the plane has only spotLight. Another problem is I cannot see any shadows appeared.

Can anyone help? Thank you very much!

func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {

    for anchor in anchors {

        let sphere = MeshResource.generateSphere(radius: 0.2)
        let simplemodel = ModelEntity(mesh: sphere, 
                                 materials: [SimpleMaterial(color: .white, 
                                                       isMetallic: false)])
        usdzEntity = simplemodel
        usdzEntity.generateCollisionShapes(recursive: true)

        // Plane primitive
        let plane: MeshResource = .generatePlane(width: 1.0,
                                                 depth: 1.0)
        let material = SimpleMaterial(color: .white,
                                 isMetallic: false)
        let entity = ModelEntity(mesh: plane,
                            materials: [material])
        usdzEntity.addChild(entity)

        usdzAnchorEntity = AnchorEntity(anchor: anchor)
        usdzAnchorEntity.addChild(usdzEntity)
        scene.addAnchor(usdzAnchorEntity)

        let lightAnchor = AnchorEntity(world: [0,0,-3])
        lightAnchor.addChild(directLight)
        lightAnchor.addChild(spotLight)
        lightAnchor.addChild(pointLight)
        scene.addAnchor(lightAnchor)
    }
}

Andy Jazz · Answer 1 · 2020-08-28T15:08:10.257

Lights in RealityKit have some peculiarities:

DirectionalLight()
- Position is not important
- Orientation is important
- Has shadows
SpotLight()
- Position is important
- Orientation is important
- Has shadows
PointLight()
- Position is important
- Orientation is not important
- No shadows

Here's how these light types look in code:

@IBOutlet var arView: ARView!

override func viewDidLoad() {
    super.viewDidLoad()
    arView.backgroundColor = .black
    
    let directLight = DirectionalLight()
    directLight.light.color = .red
    directLight.light.intensity = 10000
    directLight.position.x = 0
    directLight.orientation = simd_quatf(angle: Float.pi/5,
                                          axis: [0, 1, 0])
    directLight.shadow = .init(DirectionalLightComponent.Shadow(maximumDistance: 5, 
                                                                      depthBias: 1))
    
    let spotLight = SpotLight()
    spotLight.light.color = .green
    spotLight.light.intensity = 450000
    spotLight.position.x = -1.1
    spotLight.shadow = .init()
    
    let pointLight = PointLight()
    pointLight.light.color = .blue
    pointLight.light.intensity = 700000
    pointLight.position.x = 3.0
    // pointLight has no shadows
    // pointLight's intensity is much higher than Directional's or Spot's one


    // SPHERE
    let sphere = MeshResource.generateSphere(radius: 0.4)
    let simpleModel = ModelEntity(mesh: sphere,
            materials: [SimpleMaterial(color: .lightGray,
                                  isMetallic: true)])
    let usdzEntity = simpleModel
    usdzEntity.generateCollisionShapes(recursive: true)

    // PLANE
    let plane: MeshResource = .generatePlane(width: 2.0,
                                             depth: 2.0)
    let material = SimpleMaterial(color: .lightGray,
                             isMetallic: false)
    let entity = ModelEntity(mesh: plane,
                        materials: [material])
    entity.orientation = simd_quatf(angle: Float.pi/4,
                                     axis: [1, 0, 0])
    usdzEntity.addChild(entity)

    let usdzAnchorEntity = AnchorEntity()
    usdzAnchorEntity.addChild(usdzEntity)
    arView.scene.addAnchor(usdzAnchorEntity)

    let lightAnchor = AnchorEntity(world: [0, 0, 2.5])
    lightAnchor.addChild(directLight)
    lightAnchor.addChild(spotLight)
    lightAnchor.addChild(pointLight)
    arView.scene.addAnchor(lightAnchor)
}

Thank you very much, this is helpful! Now I have lights and shadows, the new problem is when I change the simpleModel to a VideoMaterial using `let model = ModelEntity(mesh: sphere, materials: [VideoMaterial(avPlayer: player)])`, lights and shadows both disappeared, any ideas? — xin nai wang, Aug 31 '20 at 08:50
RealityKit's `VideoMaterial` can be applied using Xcode 12+ and iOS 14+ only. Here's a link – https://developer.apple.com/documentation/realitykit/videomaterial — Andy Jazz, Aug 31 '20 at 08:51
Yeah, I know how to use VideoMaterial, the problem is VideoMaterial doesn't cast shadows... — xin nai wang, Sep 01 '20 at 00:57
RealityKit model shaded with VideoMaterial DOES cast shadows. Watch "What's new in RealityKit" WWDC 2020 – https://developer.apple.com/videos/play/wwdc2020/10612/ (timing 01:37) If you mean self-shadowing feature – you're right no one shader in RealityKit supports it (including VideoMaterial class). — Andy Jazz, Sep 01 '20 at 04:37
Sorta "stealing" post here before making my own... but how can I make shadow that act like AnchorEntity shadow that works with ibl/environment lights and not custom/generic ones? @AndyFedoroff I just want a shadow plane to work with real world lighting... ideas? — Dariusz, Jun 18 '21 at 10:48
Hi @Dariusz, what do you mean saying "Sorta "stealing" post here before making my own..." ? ))) — Andy Jazz, Jun 18 '21 at 10:53
Well its related to shadow stuff, figured I'd follow up to expand on this post for future chaps looking for help ;- ) In any case, here is post > Thank you in advance for any help ^^ https://stackoverflow.com/questions/68033965/realitykit-how-to-create-shadow-catcher-for-natural-lighting — Dariusz, Jun 18 '21 at 11:00

How can I make shadows in RealityKit for USDZ files?

1 Answers1