Converting SCNMatrix4 from swift to Objective-c

Question

I can not for the life of me figure out how to create a SCNMatrix4 from a transform in objective-c.

The swift code I'm trying to use in objective-c:

let affineTransform = frame.displayTransform(for: .portrait, viewportSize: sceneView.bounds.size)
let transform = SCNMatrix4(affineTransform)
faceGeometry.setValue(SCNMatrix4Invert(transform), forKey: "displayTransform")

I got the first and third line but I can't find anyway to create this SCNMatrix4 from the CGAffineTransform.

CGAffineTransform affine = [self.sceneView.session.currentFrame displayTransformForOrientation:UIInterfaceOrientationPortrait viewportSize:self.sceneView.bounds.size];
SCNMatrix4 trans = ??
[f setValue:SCNMatrix4Invert(trans) forKey:@"displayTransform"];

There is no SCNMatrix4Make, I tried simd_matrix4x4 but that didn't seem to work either.

Thank you

edit:

The swift code is from Apples Example project "ARKitFaceExample", this is the full code:

/*
See LICENSE folder for this sample’s licensing information.

Abstract:
Demonstrates using video imagery to texture and modify the face mesh.
*/

import ARKit
import SceneKit

/// - Tag: VideoTexturedFace
class VideoTexturedFace: TexturedFace {
    
    override func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
        guard let sceneView = renderer as? ARSCNView,
            let frame = sceneView.session.currentFrame,
            anchor is ARFaceAnchor
            else { return nil }
        
        #if targetEnvironment(simulator)
        #error("ARKit is not supported in iOS Simulator. Connect a physical iOS device and select it as your Xcode run destination, or select Generic iOS Device as a build-only destination.")
        #else
        // Show video texture as the diffuse material and disable lighting.
        let faceGeometry = ARSCNFaceGeometry(device: sceneView.device!, fillMesh: true)!
        let material = faceGeometry.firstMaterial!
        material.diffuse.contents = sceneView.scene.background.contents
        material.lightingModel = .constant

        guard let shaderURL = Bundle.main.url(forResource: "VideoTexturedFace", withExtension: "shader"),
            let modifier = try? String(contentsOf: shaderURL)
            else { fatalError("Can't load shader modifier from bundle.") }
        faceGeometry.shaderModifiers = [ .geometry: modifier]

        // Pass view-appropriate image transform to the shader modifier so
        // that the mapped video lines up correctly with the background video.
        let affineTransform = frame.displayTransform(for: .portrait, viewportSize: sceneView.bounds.size)
        let transform = SCNMatrix4(affineTransform)
        faceGeometry.setValue(SCNMatrix4Invert(transform), forKey: "displayTransform")

        contentNode = SCNNode(geometry: faceGeometry)
        #endif
        return contentNode
    }
    
}

In case anyone ever needs this, here is the extension I was missing

extension SCNMatrix4 {
    /**
     Create a 4x4 matrix from CGAffineTransform, which represents a 3x3 matrix
     but stores only the 6 elements needed for 2D affine transformations.
     
     [ a  b  0 ]     [ a  b  0  0 ]
     [ c  d  0 ]  -> [ c  d  0  0 ]
     [ tx ty 1 ]     [ 0  0  1  0 ]
     .               [ tx ty 0  1 ]
     
     Used for transforming texture coordinates in the shader modifier.
     (Needs to be SCNMatrix4, not SIMD float4x4, for passing to shader modifier via KVC.)
     */
    init(_ affineTransform: CGAffineTransform) {
        self.init()
        m11 = Float(affineTransform.a)
        m12 = Float(affineTransform.b)
        m21 = Float(affineTransform.c)
        m22 = Float(affineTransform.d)
        m41 = Float(affineTransform.tx)
        m42 = Float(affineTransform.ty)
        m33 = 1
        m44 = 1
    }
}

The Swift code you posted won't compile. I get an error on the line `let transform = SCNMatrix4(affineTransform)` because there is no initializer for `SCNMatrix4` that accepts a `CGAffineTransform`. Do you have your own extension that adds that functionality? — HangarRash, Jan 05 '23 at 01:07
@HangarRash ah thanks, I was able to find the extension in apples example project — IamRob, Jan 05 '23 at 03:00
@IamRob Hey Rob, welcome to the site! I have several thoughts/questions about this. 1. Why do you need to convert this to Objective C? Can you just keep the Swift, and call it from Objective C if necessary? 2. The same way you see them create an instance of `SCNMatrix4` by setting each of the matrix members manually, you can do yourself in ObjC. 3. IIRC `SCNMatrix4` is a C struct, and ObjC only lets you define initializers, methods, etc. for classes, you can't just definie an `-initFromCGAffineTransform:` like you see with Swift. You'll need to make a C free function that returns the struct. — Alexander, Jan 05 '23 at 03:10
@HangarRash I haven't figured out how to create an extension from that swift code yet, my brain is fried will try again tomorrow — IamRob, Jan 05 '23 at 04:37
@Alexander I didn't know I can use swift files in objc project, I will check out a youtube video on how to do that. ok I see how they are manually setting the m11, m12, etc. but I can't figure out how to access those vars in objc. — IamRob, Jan 05 '23 at 04:46
Is there a particular reason you aren’t using Swift? Usually these days people have Objective C laying around to save themselves needless rewrites, but write the new stuff in Objective C — Alexander, Jan 05 '23 at 13:50
@Alexander I've been coding in objective-c for like 9 years and haven't learned swift yet, honestly I can't stand how it looks. — IamRob, Jan 05 '23 at 22:53
Interesting, that's a perspective I haven't heard before. Most people are usually pretty quick to dismiss Objective C, particularly because of its aesthetic — Alexander, Jan 05 '23 at 23:03

HangarRash · Accepted Answer · 2023-01-05T05:08:19.783

2

To replicate the Swift extension for creating an SCNMatrix4 from a CGAffineTransform you can implement the following function:

Some .h file:

extern SCNMatrix4 SCNMatrix4FromTransform(CGAffineTransform transform);

Some .m file:

SCNMatrix4 SCNMatrix4FromTransform(CGAffineTransform transform) {
    SCNMatrix4 matrix;
    matrix.m11 = transform.a;
    matrix.m12 = transform.b;
    matrix.m21 = transform.c;
    matrix.m22 = transform.d;
    matrix.m41 = transform.tx;
    matrix.m42 = transform.ty;
    matrix.m33 = 1;
    matrix.m44 = 1;

    return matrix;
}

Then your code becomes:

CGAffineTransform affineTransform = [self.sceneView.session.currentFrame displayTransformForOrientation:UIInterfaceOrientationPortrait viewportSize:self.sceneView.bounds.size];
SCNMatrix4 transform = SCNMatrixFromTransform(affineTransform);
[f setValue:[NSValue valueWithSCNMatrix4:SCNMatrix4Invert(transform)] forKey:@"displayTransform"];

Note the use of NSValue valueWithSCNMatrix4:. This is needed to convert the struct to an object and should satisfy the use of KVC for setting the displayTransform property.

edited Jan 05 '23 at 05:08

answered Jan 05 '23 at 04:50

HangarRash

7,314
5
5
32

Thank you so much! I added that code but the third line doesn't like the type and I get this error "Sending 'SCNMatrix4' (aka 'struct SCNMatrix4') to parameter of incompatible type 'id _Nullable'" – IamRob Jan 05 '23 at 05:00
What is the type of `faceGeometry` (or `f`)? – HangarRash Jan 05 '23 at 05:03
ARSCNFaceGeometry – IamRob Jan 05 '23 at 05:05
1

I've update my answer to fix that 3rd line. This will at least allow the code to compile but I'm not positive it will work at runtime since you are trying to use key-value coding to set a private property on the `ARSCNFaceGeometry` object. – HangarRash Jan 05 '23 at 05:09
Thank you so much, that did work! I've just been trying to rewrite that swift example code above into objc. – IamRob Jan 05 '23 at 05:11

Converting SCNMatrix4 from swift to Objective-c

1 Answers1