9

In a Vulkan program, fragment shaders generally output single-precision floating-point colors in the range 0.0 to 1.0 to each red/blue/green channel, and these are then written to (blended into) the swapchain image that is then presented to screen. The floating point values are encoded into bits according to the format of the swapchain image (specified when the swapchain is created).

When I change my swapchain format from VK_FORMAT_B8G8R8A8_UNORM to VK_FORMAT_B8G8R8A8_SRGB I observe that the overall brightness of the frames is greatly increased, and also there are some minor color shifts.

My understanding of the SRGB format was that it was a lot like the UNORM format just having a different mapping of floating point values to 8-bit integers, such that it had higher color resolution in some areas and less in others, but the actually meaning of the "pre-encoded" RGB floating-point values remained unchanged.

So I'm a little suprised about the brightness increase. Is my understanding of SRGB encoding wrong? and/or is such a brightness increase is expected vs UNORM?

or maybe I have a bug and a brightness increase is not expected?

Update:

I've observed that if I use SRGB swapchain images and also load my images/textures in VK_FORMAT_B8G8R8A8_SRGB format rather than VK_FORMAT_B8G8R8A8_UNORM then the extra brightness goes away. It looks the same as if I use VK_FORMAT_B8G8R8A8_UNORM swapchain images and load my images/textures in VK_FORMAT_B8G8R8A8_UNORM format.

Also, if I put the swapchain image into VK_FORMAT_B8G8R8A8_UNORM format and then load the images/textures with VK_FORMAT_B8G8R8A8_SRGB, the frames look extra dark / almost black.

Some clarity about what is going on would be helpful.

Andrew Tomazos
  • 66,139
  • 40
  • 186
  • 319

2 Answers2

17

This is a colorspace and display issue.

Fragment shaders are assumed to be writing values in a linear RGB colorspace. As such, if you are rendering to an image that has a linear RGB colorspace (UNORM), the values your FS produces are interpreted directly. When you render to an image which has an sRGB colorspace, you are writing values from one space (linear) into another space (sRGB). As such, these values are automatically converted into the sRGB colorspace. It's no different from transforming a position from model space to world space or whatever.

What is different is the fact that you've been looking at your scene incorrectly. See, odds are very good that your swapchain's VkSurfaceFormat::colorSpace value is VK_COLOR_SPACE_SRGB_NONLINEAR_KHR.

VkSurfaceFormat::colorspace tells you how the display engine will interpret the pixel data in swapchain images you present. This is not a setting you provide; this is the display engine telling you something about how it is going to interpret the values you send it.

I say "odds are very good" that it is sRGB because, outside of extensions, this is the only possible value. You are rendering to an sRGB display device whether you like it or not.

So if you write to a UNORM image, the display device will read the actual bits of data and interpret them as if they are in the sRGB colorspace. This operation only makes sense if the data your fragment shader wrote itself is in the sRGB colorspace.

However, that's generally not how FS's generate data. The lighting computations you compute only make sense on color values in a linear RGB colorspace. So unless you wrote your FS to deliberately do sRGB conversion after computing the final color value, odds are good that all of your results have been in a linear RGB colorspace. And that's what you've been writing to your framebuffer.

And then the display engine mangles it.

By using an sRGB image as your destination, you force a colorspace conversion from linear RGB to sRGB, which will then be interpreted by the display engine as sRGB values. This means that your lighting equations are finally producing the correct results.

Failure to do gamma-correct rendering properly (including the source texture images which are almost certainly also in the sRGB colorspace, as this is the default colorspace for most image editors. The exceptions would be for things like gloss-maps, normal maps, or other images that aren't storing "colors".) leads to cases where linear light attention appears more correct than quadratic attenuation, even though quadratic is how reality works.

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
  • So if I understand your answer and the other correctly, I should use SRGB for surface format and load images/textures in UNORM ? – Andrew Tomazos Feb 27 '21 at 17:25
  • @AndrewTomazos: Most images are not in a linear colorspace, not unless you explicitly built it that way. The ones that are would be for non-picture like things: normal maps, gloss maps, etc. – Nicol Bolas Feb 27 '21 at 17:34
  • What I'm finding is that if I load sampler images into VK_FORMAT_R8G8B8A8_SRGB, putting into the host buffer the 4 byte RGBA pixels libpng gives me, and then put my surface format also into VK_FORMAT_B8G8R8A8_SRGB, the image I see on the screen when simply passing through sampled pixels to FS output - closely resembles what I see in my system image viewer. I guess we'll chalk that up as a tenative win for now. :) – Andrew Tomazos Feb 27 '21 at 17:54
5

This is gamma correction.

Using a swapchain with VK_FORMAT_B8G8R8A8_SRGB leverages the ability to to apply gamma correction as the final step in your render pipeline. This happens for you automatically behind the scenes.

That is the only place you want gamma correction to happen. Make sure your shaders are not applying gamma correction. You might see it as:

color = pow(color, vec3(1.0/2.2));

If your swapchain does the gamma correction, you do not need todo it in your shaders.

Most images are SRGB (pictures, color textures, etc). Linear images are for specific data, like a blue noise texture or heightmap.

  • Load SRGB images w/ VK_FORMAT_R8G8B8A8_SRGB
  • Load LINEAR images w/ VK_FORMAT_R8G8B8A8_UNORM

No shader conversion is required if the rules outlined above are followed.

J. Tully
  • 129
  • 1
  • 9
  • examples of non-srgb images would be normal maps, noise maps - these should be loaded with the unorm suffix. Any color related maps that the user will see should be saved as srgb - these should be loaded with the srgb suffix. – J. Tully May 16 '21 at 11:34
  • If I load an image with srgb and then pass it to fragment shader, does vulkan automatically transfer the image from srgb nonlinear space into linear space? Or I still need to do that explicitly, such as pow(color, vec3(2.2)) . – wangsy Sep 12 '21 at 10:32
  • @wangsy I revised my answer to be more specific and clear. – J. Tully Sep 17 '21 at 06:18