4

I want to create an AR experience with a full screen video with AR elements, and in addition a small view at the bottom of the screen to show some relevant 3D objects. In the past, I used to implement such features with an ARSCNView for the AR part, and a SCNView (SceneKit view) for the little preview part.

Is there a way I can do that with RealityKit for those 2 views?

I understand that RealityKit needs an ARView to render, which means I would need 2 ARViews on my screen, with different cameraMode settings:

let arView = ARView(frame: self.view.frame,
                    cameraMode: .ar,
                    automaticallyConfigureSession: true)
view.addSubview(arView)
        
let newAnchor = AnchorEntity(world: [0, 0, -1])
let newBox = ModelEntity(mesh: .generateBox(size: 0.5))
newAnchor.addChild(newBox)
arView.scene.anchors.append(newAnchor)
        
        
        
let arView2 = ARView(frame: CGRect(x:50,y:300,width: 200,height:200),
                     cameraMode: .nonAR,
                     automaticallyConfigureSession: true)
view.addSubview(arView2)
        
let newAnchor2 = AnchorEntity(world: [0, 0,-1])
let newBox2 = ModelEntity(mesh: .generateBox(size: 0.5))
newAnchor2.addChild(newBox2)
arView2.scene.anchors.append(newAnchor2)

arView shows the video feed with the newBox as expected.

My problem is that arView2 shows the camera feed although I expected a black view with only the newBox2. In addition this camera feed is distorted (it seems to be the contents of arView resized to fit arView2)

What am I missing?

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
Gogo123
  • 655
  • 1
  • 4
  • 11

1 Answers1

0

Unlike ARKit, in RealityKit 2.0 you cannot simultaneously run two ARSessions (I mean a face config as primary setting, and a world config as a secondary setting), just a single one. Therefore, the running session of the first ARView must be used by both views.

arView2.session = arView.session

arView2.environment.background = .color(.systemIndigo)
Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
  • 2
    thanks Andy for your help. Unfortunately, arView2 still shows the AR video in its background although I would need to a transparent background. Any idea? – Gogo123 Sep 07 '22 at 08:53
  • 2
    self.arView2.environment.background = .color(.clear) only works if I also do self.arView.environment.background = .color(.green) which is not what I want. I need the video feed in arView and no background in arView2 – Gogo123 Sep 07 '22 at 09:08
  • 1
    Hi @Gogo123, you're right, RealityKit renders either both views with BG-video, or both views with VR. Looks like a bug (used Xcode 14.0 beta 6). – Andy Jazz Sep 07 '22 at 10:16