Is it possible to have an animated background texture in SceneKit?

Question

I mean animating the scene background (scene.background.contents). Not a texture placed on an object.

I currently have a .exr file as the background in my project, but I would like to have an animated background running in a loop. My first idea was creating an array of exr images, turning that into an animation and then using that animation as a material's diffuse texture. But, it isn't working.

I only created 3 starting frames to test (background_0, 1 and 2)
But, when the app loads I can see the background for half a second before it turns black.

Did I do something wrong? or maybe I'm approaching this issue in a wrong way and there's an easier way to achieve this effect??

This is the code I currently have for the background, any help is very much appreciated:

// load the image sequences: .exr
var imageArray: [UIImage] = []
for i in 0...2 { // afor now 3 frames long, 0, 1 and 2
  if let image = UIImage(named: "background_\(i).exr") { // we assume their names: "background_0.exr", "background_1.exr", etc.
      imageArray.append(image)
      print(image)
  }
}
print(imageArray)

// Create texture animation
let bg_animation = CAKeyframeAnimation(keyPath: "contents")
bg_animation.calculationMode = .discrete
bg_animation.duration = 5 // animation's duration
bg_animation.values = imageArray.map { $0.cgImage as Any }
bg_animation.repeatCount = .infinity // animation repeats forever.
//bg_animation.autoreverses = true // the animation reverses

print("hi, this is the bg animation: ")
print(bg_animation)

// Create the animation using the array
backgroundAnimation = SCNMaterialProperty(contents: bg_animation)

print("hi, this is the SCNMaterial property animation: ")
print(backgroundAnimation)

// Create the material and asign the animation as its content
let backgroundMaterial = SCNMaterial()
backgroundMaterial.setValue(backgroundAnimation, forKey: "contents")
backgroundMaterial.locksAmbientWithDiffuse = true

print("This is background Material")
print(backgroundMaterial)
print(backgroundMaterial.animationKeys)
print(backgroundMaterial.diffuse)

// Assign the material to the scene background content
scene.background.contents = backgroundMaterial

print("Asign material as background")
print(scene.background.contents)

I tried:
create an array of images, turning that into animation, that animation using it as a texture and assigning it as the background content.
Each frame works on their own but not as an array it seems.

Answer 1

Try one of the following approaches (UIKit vs SwiftUI):

UIKit version

The simplest solution to create an animated SCNScene's background based on OpenEXR image sequence, is to implement the renderer(_:updateAtTime:) delegate method.

import UIKit
import SceneKit

extension ViewController: SCNSceneRendererDelegate {
    
    func renderer(_ renderer: SCNSceneRenderer,
           updateAtTime time: TimeInterval) {
        scene.background.contents = imageArray[Int(time / 5) % 3]
    }
}

---

class ViewController: UIViewController {
    
    @IBOutlet var sceneView: SCNView!
    var imageArray: [UIImage] = []
    let scene = SCNScene()

    override func viewDidLoad() {
        super.viewDidLoad()
        
        sceneView.delegate = self
        sceneView.scene = scene
        sceneView.isPlaying = true
        sceneView.allowsCameraControl = true
        
        for frame in 0...2 {
            DispatchQueue.main.async { [self] in
                if let image = UIImage(named: "bg_\(frame).exr") {
                    self.imageArray.append(image)
                }
            }
        }
    }
}

SwiftUI version

The SwiftUI version will additionally require you to implement a coordinator.

import SwiftUI
import SceneKit

struct ContentView : View {
    var body: some View {
        SceneKitView().ignoresSafeArea()
    }
}

---

struct SceneKitView : UIViewRepresentable {        
    @State var imageArray: [UIImage] = []
    let sceneView = SCNView(frame: .zero)
    let scene = SCNScene()
    
    func makeCoordinator() -> Coordinator { Coordinator(self) }
    
    final class Coordinator: NSObject, SCNSceneRendererDelegate {
        var control: SceneKitView
        init(_ control: SceneKitView) { self.control = control }
        
        func renderer(_ renderer: SCNSceneRenderer, 
               updateAtTime time: TimeInterval) {
            control.scene.background.contents = 
                                     control.imageArray[Int(time / 5) % 3]
        }
    }
    func updateUIView(_ view: SCNView, context: Context) {
        for frame in 0...2 {
            DispatchQueue.main.async {
                if let image = UIImage(named: "bg_\(frame).exr") {
                    imageArray.append(image)
                }
            }
        }
    }
    func makeUIView(context: Context) -> SCNView {
        sceneView.delegate = context.coordinator
        sceneView.scene = scene
        sceneView.isPlaying = true
        sceneView.allowsCameraControl = true
        return sceneView
    }
}