3

I have a trivial macOS app that has an MTKView--it is just an MTKView inside a window. The fragment shader always does:

return float4(1, 0, 0, 0.01)

That's red with 1% alpha.

The CAMetalLayer belonging to the MTKView is non-opaque, as is the window:

_mtkView.layer.opaque = NO;
_mtkView.window.opaque = NO;
_mtkView.window.backgroundColor = [NSColor clearColor];
_mtkView.window.hasShadow = NO;

When I put this window over another app's window, I would expect the other app's pixels to show through unchanged. Instead, I see that black pixels turn red and white pixels are mostly unchanged.

In the screenshot below, the triangle has color float4(1,0,0,0.01) and there's a half-black half-white image in another window showing through it:

Screenshot showing the problem

I would have expected black ever-so-slightly tinged with red, not bright red, in the left half of the triangle.

If my fragment shader is changed to do this:

return float4(1, 0, 0, 0.001);

Then it is completely transparent: pixels from other windows show through completely unchanged. This makes me think I'm hitting a not-well-tested code path in the window server, but I'm hoping there's something I can do to make it work.

What I tried

  • Changing the appearance of the NSThemeFrame (had no effect)
  • Turning on presentsWithTransaction (no effect)
  • The GPU debugger confirms the drawable has the expected value
George
  • 4,189
  • 2
  • 24
  • 23
  • Make sure that the layer's `compositingFilter` and `backgroundFilters` are `nil`. Does setting the view's `framebufferOnly` to false change anything? – Ken Thomases Jul 16 '18 at 11:33
  • compositingFilter and backgroundFilters are nil and empty for both the CAMetalLayer and its superlayer. Setting framebufferOnly=NO on the CAMetalLayer has no effect. – George Jul 16 '18 at 17:23
  • As an experiment, what happens if you enclose your Metal view in a layer-back view that uses normal Cocoa drawing to draw that half-black, half-white image? That is, eliminate compositing of windows with each other and just verify view layer compositing. Similarly, try an experiment with no Metal, just a layer-backed view that fills with clear and draws the mostly-transparent triangle and see how that composites. – Ken Thomases Jul 16 '18 at 18:20
  • That was a good idea. It does not composite correctly, and shows the same symptoms. – George Jul 17 '18 at 00:22
  • …and it *does* composite correctly when using a layer-hosting view with a non-opaque layer whose contents are a NSImage with a transparent triangle. – George Jul 17 '18 at 00:28
  • I wonder if it's an issue with the graphics drivers or GPU. Are you able to test on different hardware? – Ken Thomases Jul 17 '18 at 02:06
  • 2015 imac, imac pro, and 2016 macbook pro all give the same result. Time for a radar? – George Jul 17 '18 at 06:39
  • Sounds like it. – Ken Thomases Jul 17 '18 at 12:22
  • Apple responded and said that I should have used premultiplied alpha, and the proper fragment function would be `return float4(0.01, 0, 0, 0.01)` for 1% red. Taking that approach with yellow still shows funny blending. This image has a layer with backgroundColor 1,1,0,0.1 on top and MTKView with 0.1,0.1,0,0.1 on the bottom: https://imgur.com/a/jiv5qAs – George Jul 18 '18 at 02:26
  • I think I've got it now. It also has to be converted to linear color space by the fragment function after premultiplying alpha. – George Jul 18 '18 at 02:36
  • Huh. Very odd. In general, fragment shaders don't have to output premultiplied alpha to get proper blending. Did they say it's specific to cross-window blending? Also, fragment shaders always work with linear RGB, regardless of source texture or render target pixel formats. Did you mean you were getting sRGB values from some other place and needed to convert those to linear? Anyway, glad you got it working. You can self-answer. – Ken Thomases Jul 18 '18 at 03:50

1 Answers1

5

Apple wrote:

The CoreAnimation compositor requires all of its inputs be premultipled by alpha. See these references for more information: https://en.wikipedia.org/wiki/Alpha_compositing https://blogs.msdn.microsoft.com/shawnhar/2009/11/06/premultiplied-alpha/

When RGB are premultipled by alpha, then any RGB component that is > the alpha component is undefined through the hardware’s blending.

Modifying your fragment shader to return (0.1, 0, 0, 0.1), which is “0.1 alpha, full red” produces the expected results.

Indeed, this fixed my issue. The relevant commit is https://github.com/gnachman/iTerm2/commit/70a93885b1f67bda843ba546aca0eca67b750088 where I modified the program to draw to an offscreen texture and then copy it to the drawable while multiplying the red, green, and blue components by alpha.

For some reason that I cannot understand, my test program also required conversion to linear color space.

George
  • 4,189
  • 2
  • 24
  • 23
  • Can I ask where you found that documentation? I think I'm having a similar issue but can't find any mention about this in Apple's Metal docs. – choxi Dec 02 '19 at 02:53
  • I filed a radar and they responded with that message. It would be nice if they documented this! – George Dec 02 '19 at 06:26