Set CALayer as SCNMaterial's diffuse contents

Question

I've been searching all over the internet over the past couple of days to no avail. Unfortunately, the apple documentation about this specific issue is vague and no sample code is available (at least thats what I found out). What seems to be the issue you may ask...

I'm trying to set a uiview's layer as the contents of the material that is used to render an iPhone model's screen (Yep, trippy :P ). The iPhone's screen's UV mapping is set from 0 to 1 so that no issue persists in mapping the texture/layer onto the texels.

So, instead of getting this layer to appear rendered on the iPhone, same as left image, Instead, I get this rendered onto the iPhone like right image

Correct Render Incorrect Render

Also note, that when I set a breakpoint and debug the actual iPhone node and view it in Xcode, a completely different render is shown and the layer gets half-fixed when I continue execution:

Now then... HOW do I fix this issue??? I've tried playing with the diffuse's contents transform matrix but nothing gets fixed. I've also tried resizing the UIView to 256x256 (since the UV seems to be 256x256 as shown in blender - the 3d modelling package), but that doesn't fix anything.

Here is the code for the layer:

UIView *screen = [[UIView alloc] initWithFrame:self.view.bounds];
screen.backgroundColor = [UIColor redColor];

UIView *temp = [[UIView alloc] initWithFrame:CGRectMake(0, 0, screen.bounds.size.width, 60)];
temp.backgroundColor = [UIColor colorWithRed:0 green:(112.f/255.f) blue:(235.f/255.f) alpha:1];

UILabel *label = [[UILabel alloc] initWithFrame:CGRectInset(temp.bounds, 40, 0)];
label.frame = CGRectOffset(label.frame, 40, 0);
label.textColor = [UIColor colorWithRed:0 green:(48.f/255.f) blue:(84.f/255.f) alpha:1];
label.text = @"Select Track";
label.font = [UIFont fontWithName:@"HelveticaNeue-Light" size:30];
label.minimumScaleFactor = 0.001;
label.adjustsFontSizeToFitWidth = YES;
label.lineBreakMode = NSLineBreakByClipping;
[temp addSubview:label];

UIView *separator = [[UIView alloc] initWithFrame:CGRectMake(0, temp.bounds.size.height - 2, temp.bounds.size.width, 2)];
separator.backgroundColor = [UIColor colorWithRed:0 green:(48.f/255.f) blue:(84.f/255.f) alpha:1];
[temp addSubview:separator];
[screen addSubview:temp];
screen.layer.contentsGravity = kCAGravityCenter;

Edit
What's even weirder is that if I capture a UIImage of the view using:

- (UIImage *) imageWithView:(UIView *)view
{
    UIGraphicsBeginImageContextWithOptions(view.bounds.size, view.opaque, 0.0);
    [view.layer renderInContext:UIGraphicsGetCurrentContext()];

    UIImage * img = UIGraphicsGetImageFromCurrentImageContext();

    UIGraphicsEndImageContext();

    return img;
}

and use that as the diffuse's content... everything works out perfectly fine?! It's really weird and frustrating since the image's size is exactly the same as the uiview's...

Edit 2

I ended up just using an image of the view as the texture, which makes things much more static than I needed. I won't set this as the answer because I'll still be waiting for a correct fix to this issue even if it in a long time. So, if you have an answer and this topic has been opened for a long time, please bump it if you can. The documentation on this section is just so poor.

Can we see how you're setting up the SCNScene and SCNView? Does it work when you use a solid color as the material instead of a CALayer? — Hal Mueller, Mar 24 '16 at 03:01
@HalMueller I'm using the default code that comes when you create a SceneKit application, and only change the name of the scene as well as the nodes. The CALayer's code is shown [here](https://ghostbin.com/paste/g2ds7). It will work with a solid colour since the layer is, in reality, mapped in a sense that allows for it to cover up the entire "iPhone screen". It seems that the mapping makes it so that the layer overflows outside the texels' regions... but when I use a texture image in ib, it fits perfectly and looks right. — Brave Heart, Mar 24 '16 at 04:54
I would suggest to try and add a `shouldRasterize = YES;` to your `CALayer`, as well as calling `[SCNTransaction flush];` right after you assign the `CALayer` to the contents — Coldsteel48, Dec 06 '16 at 08:48
@ColdSteel. The desire to use a CALayer is because of the Animation inherent to CALayer. In other words, the whole reasons to use a CALayer as a texture is because it's uniquely animatable, in many good, fun ways. If relying on shouldRasterize, one might as well use a bitmap, since a CALayer that's rasterised is a bitmap. — Confused, Dec 06 '16 at 18:55
@Confused AFAIK it is still animates itself, by creating bitmap sequences under the hood — Coldsteel48, Dec 06 '16 at 19:17
You lose ALL the performance advantages of using Core Animation if it's attempting to recreate bitmaps every frame of animation, as this becomes similar to Core Graphics drawing into a context before blitting to the screen. — Confused, Dec 06 '16 at 19:51
@ColdSteel the reason to use CALayers as textures is to have the most performant possible animatable textures. Similarly, if SpriteKit were used as a texture in SceneKit. CALayers have some abilities that SpriteKit does not, hence the attraction to them. — Confused, Dec 06 '16 at 19:52
I suspect, you are setting up your view in `viewDidLoad`. If yes, you are getting wrong size of your `self.view`. — Blind Ninja, Mar 21 '17 at 07:30

Bobjt · Answer 1 · 2020-01-23T13:00:14.597

6

New post on an old thread, but this day-in-age, it's possible to set the UIView itself as SCNMaterialProperty (diffuse) contents. Intention to support this feature is communicated directly from SceneKit engineering at Apple, though the documentation has not yet been updated to reflect it.

To tied back to the original post, do not set a UIView.layer as material property contents; instead set contents to the UIView itself.

[Update: according to Lance's comment below, support for views may be getting worse rather than getting better.]

edited Jan 23 '20 at 13:00

answered Oct 18 '17 at 20:08

Bobjt

4,040
1
29
29

1

That's great and I verified it works. Unfortunately, it doesn't support transparency, though. – Ortwin Gentz Jan 12 '18 at 09:27
Yikes! did you file a bug? – Bobjt Jan 12 '18 at 17:44
@Bobjt As of 2020 UIView isn't fully supported yet. I keep getting **ARKit_4[20429:4746699] [Animation] +[UIView setAnimationsEnabled:] being called from a background thread. Performing any operation from a background thread on UIView or a subclass is not supported and may result in unexpected and insidious behavior**: https://stackoverflow.com/questions/59446393/arkit-uiview-setanimationsenabled-performing-any-operation-from-a-background – Lance Samaria Jan 22 '20 at 22:52
I'll delete this post if support is getting worse. Who's calling setAnimationsEnabled, out of curiosity? – Bobjt Jan 23 '20 at 13:01
1

@Bobjt I'm following this answer: https://stackoverflow.com/a/48712156/4833705. The diffuse.contents property is what's causing the error. I don't think you should delete the answer as of yet. I'm using one of my TSIs to see what Apple techs have to say about the issue. I'll update you once I get an answer from them – Lance Samaria Jan 24 '20 at 04:09

score 4 · Answer 2 · answered Dec 12 '16 at 03:09

The SceneKit docs pretty strongly suggest that, while there are cases where you can use animated CALayers as material content, that doesn't include UIView layers:

SceneKit cannot use a layer that is already being displayed elsewhere (for example, the backing layer of a UIView object).

That suggests that if you want to make animated content for your material, you're better off with either Core Animation used entirely on its own or SpriteKit.

Set CALayer as SCNMaterial's diffuse contents

Correct Render Incorrect Render

2 Answers2

Linked