7

I want to start by saying that I am new to SceneKit and playing with AR environments on iOS.

I have a 3D model object, a van, as part of a Blender file(.blend) --> it can be found here.

I am exporting it as a .glb file from Blender --> it can be found here.

Then I open the .glb file in Reality Converter app, and export it as a .usdz file --> it can be found here.

The issue is when I preview the object in XCode and also when I place it in AR, the object colours are wrong.

Is there a problem with the Blender object? If so, what should I change to it so that the colours are exported correctly?

enter image description here

enter image description here

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
Laur Stefan
  • 1,519
  • 5
  • 23
  • 50

1 Answers1

10

About Blender, Maya and Xcode

Unfortunately, the issue you mentioned here is a great old issue in Blender. Indeed, Blender outputs wrong (a.k.a. inconsistent) shaders' colors, due to the fact that Blender operates in linear color space (not sRGB) of shader colors. There are 2 possible solutions in this case. The first one is quite radical - to use a real PRO software, like Autodesk Maya 2023 with USD plugin (because it takes a decent amount of time to learn Maya), in which exported shaders' colors are reproduced very accurately in correct color space – you must reassign all the materials from scratch. The second one – fix colors in Xcode's inspector manually, or programmatically.

From Maya you can export your model as binary or ASCII .usd file format (make sure that MayaUSD plugin is loaded). Then use Reality Converter, to convert the resulting model in .usdz archive.

Xcode solution

Load your van.usdz model into Xcode. Rename model's nodes in hierarchy for convenience.

enter image description here

Very important!

Select Body_Orange node and go to Attributes Inspector tab. Delete Colors attribute.

Colors attribute keeps .usdz colour schema.

enter image description here

Then in Material Inspector assign a needed Diffuse color of physicallyBased shader.

enter image description here

Also, you can do it programmatically. Here's a SwiftUI sample code for SceneKit:

import SwiftUI
import SceneKit
...

func makeUIView(context: Context) -> SCNView {

    let sceneView = SCNView(frame: .zero)
    sceneView.scene = SCNScene(named: "van.usdz")
    
    let orangeBody = sceneView.scene?.rootNode.childNodes[0].childNodes[0]
        .childNodes[0].childNode(withName: "Body_Orange", recursively: true)

    orangeBody?.geometry?.firstMaterial?.diffuse.contents = 
          UIColor(red: 218/255, green: 154/255, blue: 93/255, alpha: 255/255)

    let blueBody = sceneView.scene?.rootNode.childNodes[0].childNodes[0]
        .childNodes[0].childNode(withName: "Body_Blue", recursively: true)

    blueBody?.geometry?.firstMaterial?.diffuse.contents = 
          UIColor(red: 58/255, green: 150/255, blue: 207/255, alpha: 255/255)

    sceneView.backgroundColor = .black
    sceneView.autoenablesDefaultLighting = true
    sceneView.allowsCameraControl = true
    sceneView.showsStatistics = true

    return sceneView
}

And here's a SwiftUI sample code for RealityKit version:

import SwiftUI
import RealityKit
...

func makeUIView(context: Context) -> ARView {
    
    let arView = ARView(frame: .zero)
    arView.backgroundColor = .black
    
    let vanModel = try! ModelEntity.load(named: "van.usdz")
     
    let orangeBodyPart = vanModel.children[0].children[0].children[0]
                                 .children[0].children[0] as! ModelEntity
    let colorOrange = UIColor(red: 218/255, green: 154/255, 
                             blue: 93/255, alpha: 255/255)
    orangeBodyPart.model?.materials = [SimpleMaterial(color: colorOrange, 
                                                 isMetallic: false)]
    
    let blueBodyPart = vanModel.children[0].children[0].children[0]
                               .children[0].children[1] as! ModelEntity
    let colorBlue = UIColor(red: 58/255, green: 150/255, 
                           blue: 207/255, alpha: 255/255)
    blueBodyPart.model?.materials = [SimpleMaterial(color: colorBlue, 
                                               isMetallic: false)]

    let anchor = AnchorEntity(world: [0, 0,-5])
    vanModel.setParent(anchor)
    arView.scene.anchors.append(anchor)        
    return arView
}

enter image description here

A few words on poly count and retopology in Maya

I would like to make one remark about the polygons' count in your model. Such a simple model can be made having 2K to 5K polygons. But your model has over 50,000 polygons, which is a tremendous number. Your hi-poly model will consume a lot of resources (CPU, GPU, memory), thus, drop frame is possible (58 fps, as you can see in the picture).

So, what can you do about it? Ideally, it's a 3D modeler's job to conceive and create low-poly models for games and mobile platforms – it's really a precious skill. Modifying an already finished mesh (saved to distribution formats like .fbx or .usdz) is often a bad idea. However, Autodesk Maya for example, has a retopology tool (Mesh –> Retopologize) that helps you significantly reduce the number of polygons using advanced algorithm. Watch the official video (Time - 00:50).

enter image description here

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
  • 2
    So, what would be the steps? Save the model in what format so that it can be open in Maya and further export it as usdz? – Laur Stefan Apr 10 '22 at 23:22
  • Import your car model as `.usd` or `.dae` or `.fbx` or `.obj` into Maya, then assign new Maya shaders with correct colors (create a new UV map if needed), and then export a model as `.usd` or `.fbx`. – Andy Jazz Apr 10 '22 at 23:28
  • Then use USDZ conversion tools – https://stackoverflow.com/questions/50846627/how-to-create-usdz-file-using-xcode-converter/50867018#50867018 – Andy Jazz Apr 10 '22 at 23:29
  • 2
    @ andy-jazz, thank you , I will test it now. Also, sorry to bother you, how can one reduce the polygon count? Would it still look the same? I am asking because the model is not done by me. – Laur Stefan Apr 14 '22 at 10:56
  • 2
    Ideally, it's a 3D modeler's job to conceive and create a low-poly models (it's a cool skill). Modifying an already finished mesh is always bad. But, Maya for example, has a retopology tool (Mesh –> Retopologize.). https://www.youtube.com/watch?v=0LKrSvMatpA (Time - 00:55). – Andy Jazz Apr 14 '22 at 11:05
  • Take into consideration, AR and VR scenes on iOS must have not greater than 100K polygons. https://developer.apple.com/documentation/arkit/adding_visual_effects_in_ar_quick_look_and_realitykit – Andy Jazz Apr 14 '22 at 11:11