0

I'm trying to implement Light Estimation with ARKit, but with no luck. What I did is run a session with

configuration.isLightEstimationEnabled = true

and

arSceneView.scene                        = SCNScene()
arSceneView.autoenablesDefaultLighting   = true
arSceneView.automaticallyUpdatesLighting = true

Is there something else I need to do to make it work? There are no other lights in the scene, just autoenablesDefaultLighting. I'm using iPhone 6s, can it be too old to run this functionality?

Damian Dudycz
  • 2,622
  • 19
  • 38

1 Answers1

7

Depending on what you actually need to achieve the following may help.

autoEnablesDefaultLighting is a Boolean value that determines whether SceneKit automatically adds lights to a scene or not.

By default this is set as false meaning that:

the only light sources SceneKit uses for rendering a scene are those contained in the scene graph.

If on the other hand, this is set to true:

SceneKit automatically adds and places an omnidirectional light source when rendering scenes that contain no lights or only contain ambient lights.

This is located from the position of the camera and pointing in the direction of the camera.

One issue that Mark Daws noted in his excellent article is that:

The light has a changing direction, so as you walk around an object it will always look like the light is coming from your point of view (like you are holding a torch infront of you) which isn’t the case normally, most scenes have static lighting so your model will look unnatural as you move around.

isLightEstimationEnabled on the other hand:

provides light estimates in the lightEstimate property of each ARFrame it delivers. If you render your own overlay graphics for the AR scene, you can use this information in shading algorithms to help make those graphics match the real-world lighting conditions of the scene captured by the camera. The ARSCNView class automatically uses this information to configure SceneKit lighting.

What this means, is that if you dim the lights in your room for example, and want to apply these lighting conditions on your virtual objects to make them more realistic, this is what you want to use; since with this information we can take the lightEstimate from every frame and modify the intensity of the lights in our scene to mimic the ambient light intensity of the real world itself.

You can get details of lightingEstimate by setting:

configuration.isLightEstimationEnabled = true

And then using the following callback:

//--------------------------
// MARK: - ARSCNViewDelegate
//--------------------------

extension ViewController: ARSCNViewDelegate{

    func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {

        guard let lightEstimate = self.augmentedRealityView.session.currentFrame?.lightEstimate else { return }

        let ambientLightEstimate = lightEstimate.ambientIntensity

        let ambientColourTemperature = lightEstimate.ambientColorTemperature


        print(
            """
            Current Light Estimate = \(ambientLightEstimate)
            Current Ambient Light Colour Temperature Estimate = \(ambientColourTemperature)
            """)


        if ambientLightEstimate < 100 { print("Lighting Is Too Dark") }

    }
}

You then need to do something with the values returned and apply them to your sceneLights.

Putting it into practice therefore a very basic example might look like so:

//--------------------------
// MARK: - ARSCNViewDelegate
//--------------------------

extension ViewController: ARSCNViewDelegate{

    func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {

        //1. Get The Current Light Estimate
        guard let lightEstimate = self.augmentedRealityView.session.currentFrame?.lightEstimate else { return }

        //2. Get The Ambient Intensity & Colour Temperatures
        let ambientLightEstimate = lightEstimate.ambientIntensity

        let ambientColourTemperature = lightEstimate.ambientColorTemperature

        print(
            """
            Current Light Estimate = \(ambientLightEstimate)
            Current Ambient Light Colour Temperature Estimate = \(ambientColourTemperature)
            """)

        if ambientLightEstimate < 100 { print("Lighting Is Too Dark") }

        //3. Adjust The Scene Lighting
        sceneLight.intensity = ambientLightEstimate
        sceneLight.temperature = ambientColourTemperature
    }
}

class ViewController: UIViewController {

    //1. Create A Reference To Our ARSCNView In Our Storyboard Which Displays The Camera Feed
    @IBOutlet weak var augmentedRealityView: ARSCNView!
    @IBOutlet weak var takeSnapshotButton: UIButton!

    //2. Create Our ARWorld Tracking Configuration
    let configuration = ARWorldTrackingConfiguration()

    //3. Create Our Session
    let augmentedRealitySession = ARSession()

    //4. Create Our Light
    var sceneLight: SCNLight!

    //-----------------------
    // MARK: - View LifeCycle
    //-----------------------

    override func viewDidLoad() {

        //2. Setup The ARSession
        setupARSession()

        //2. Generate Our Scene
        generateScene()

        super.viewDidLoad()

    }

    override func didReceiveMemoryWarning() { super.didReceiveMemoryWarning() }

    //-------------------------
    // MARK: - Scene Generation
    //-------------------------

    /// Creates An SCNNode & Light For Our Scene
    func generateScene(){

        //1. Create An SCNNode With An SCNSphere Geometry
        let sphereNode = SCNNode()
        let sphereGeometry = SCNSphere(radius: 0.2)
        sphereGeometry.firstMaterial?.diffuse.contents = UIColor.cyan
        sphereNode.geometry = sphereGeometry
        sphereNode.position = SCNVector3(0, 0, -1.5)

        //2. Create Our Light & Position It
        sceneLight = SCNLight()
        sceneLight.type = .omni

        let lightNode = SCNNode()
        lightNode.light = sceneLight
        lightNode.position = SCNVector3(0,0,2)

        //3. Add The Node & Light To Out Scene
        self.augmentedRealityView.scene.rootNode.addChildNode(sphereNode)
        self.augmentedRealityView.scene.rootNode.addChildNode(lightNode)

    }

    //-----------------
    //MARK: - ARSession
    //-----------------

    /// Sets Up The ARSession
    func setupARSession(){

        //1. Set The AR Session
        augmentedRealityView.session = augmentedRealitySession
        augmentedRealityView.delegate = self
        configuration.isLightEstimationEnabled = true
        augmentedRealityView.automaticallyUpdatesLighting = false
        augmentedRealityView.autoenablesDefaultLighting = false
        augmentedRealityView.session.run(configuration, options:[.resetTracking, .removeExistingAnchors])

    }
}

Hope it helps...

BlackMirrorz
  • 7,217
  • 2
  • 20
  • 31
  • Ok so I understand that I need to create and update my own lights manually? I was thinking, that it can work automatically with just standard lights if I use autoenablesDefaultLighting = true; automaticallyUpdatesLighting = true. Can't this be done this way? Because these variables names would suggest so. – Damian Dudycz Jul 16 '18 at 05:41
  • If you want to control specific lighting then you need to create your own lights. Otherwise you can use augmentedRealityView.automaticallyUpdatesLighting = true augmentedRealityView.autoenablesDefaultLighting = true – BlackMirrorz Jul 16 '18 at 05:51
  • Ok but I did use these values as true, with no other lights added. A default lightning is added to the scene but it's not updated to match the environment. Why can this be? – Damian Dudycz Jul 16 '18 at 05:52
  • The light has a changing direction, so as you walk around an object it will always look like the light is coming from your point of view (like you are holding a torch infront of you) which isn’t the case normally, most scenes have static lighting so your model will look unnatural as you move around. – BlackMirrorz Jul 16 '18 at 06:01
  • Ok so I guess I just add own lights as you shown. I already tested it and it works fine. Thank you. – Damian Dudycz Jul 16 '18 at 06:04