0

I'm writing a PBR shader and I'm trying to determine the correct way to generate the alpha value that is output for the image. The goal is to have a final image that is 'premultiplied alpha', so it can be used with another Over composite operation later on (say, compositing it over another background image). This all works great, except for the case of specular highlights on transparent surfaces.

Reading this article: https://google.github.io/filament/Filament.md.html#lighting/transparencyandtranslucencylighting

In particular:

Observe a window and you will see that the diffuse reflectance is transparent. On the other hand, the brighter the specular reflectance, the less opaque the window appears. This effect can be seen in figure 63: the scene is properly reflected onto the glass surfaces but the specular highlight of the sun is bright enough to appear opaque.

They also include the code snippet:

// baseColor has already been premultiplied
vec4 shadeSurface(vec4 baseColor) {
    float alpha = baseColor.a;

    vec3 diffuseColor = evaluateDiffuseLighting();
    vec3 specularColor = evaluateSpecularLighting();    

    return vec4(diffuseColor + specularColor, alpha);
}

When rendering glass the metallic level would be 0, so it's not pulling from the baseColor at all, and just using the specular level. This all makes sense and means specular highlights can still occur even on the transparent glass, as per the above quote. I don't want to just multiply by alpha at the end, since my baseColor texture would already be a premultiplied alpha image, such as something containing a decal. For example let's say I'm drawing a pane of glass with a sticker on it that's the letter 'A'.

My question is: for a pixel that has a specular highlight on a transparent portion of the glass, what should the alpha value be for compositing to work downstream? The alpha would be 0 according to the above code, which would make an Over operation later on behave poorly. I say this since a property of pre-multiplied alpha is that max(R,G,B) <= A, but this specular highlight would have max(R,G,B) > A. This would result in the over operation being additive between the specular highlight and the background it's being composited over, blowing out the final image. Should I set the alpha to max(specular, alpha), to give a more useful alpha value for these pixels? Is there a more 'correct' way to handle this? Thanks

mb13
  • 139
  • 12

1 Answers1

1

The idea behind the pre-multiplied approach is we assume that the alpha/opacity only affects the diffuse (i.e. the light that goes inside the material/surface), whereas the specular is assumed to be unaffected (at least for the case of a dielectric, the light is simply reflected off the surface un-tinted).

In essence, you should have :

return vec4(diffuseColor * alpha + specularColor, alpha);

Where the alpha is 'pre-multiplied' with the diffuse colour, while the specular colour is left at full intensity (as it should).

Your blend mode will also need to follow suit : off the top of my head one/one for the source/destination colours since the colour will need to be rendered in an additive fashion.

Althar93
  • 147
  • 6
  • Thanks for your response! The result from the initial render is as you say, with the specular not being multiplied by the alpha. The issue I'm facing though is the alpha written to the buffer, and using that buffer for a another operation downstream, such as a Composite Over operation. – mb13 Apr 12 '21 at 18:14
  • I am not 100% sure what you mean by 'Composite Over' & without knowing more about your pipeline it is difficult to advise. Regardless of what you do with alpha, and what you write out, the blend mode ONE/ONE for the pre-multiplied alpha should still hold though. – Althar93 Apr 12 '21 at 21:59
  • Right, so let's say I'm creating a still image I want to give to someone working in AfterEffects. It's going to be used to be put over a backplate. AfterEffects will expect the entire image is already in pre-multiplied alpha format. However the pixels where the specular highlight on the glass is is an issue right now, since they don't follow the rule of premultiplied alpha, which is max(R,G,B) <= A. – mb13 Apr 13 '21 at 22:07
  • If you are dealing with PBR, then you should presumably have a HDR pipeline/render target & a tonemapper at the end of it all to bring the image in the [0;1] range (unless you can process HDR images directly in AfterEffects, I have no experience with that software) ; the specular need not follow the max(R,G,B) <= A. – Althar93 Apr 13 '21 at 23:51