Deferred shading anti-aliasing

Question

Many people note that MSAA does not work with deferred shading. Why is that? It sounds okay in my head.

According to wikipedia:

... due to separating the lighting stage from the geometric stage, hardware anti-aliasing does not produce correct results anymore since interpolated subsamples would result in nonsensical position, normal, and tangent attributes.

Can you explain that?

What are the other AA alternatives I have?

Answer 1

Multisampling does work with deferred rendering. Deferred rendering simply changes the price you pay for it.

Multisampling is based on the general idea that you only need to execute the fragment shader once for all of the samples covered by a particular triangle's rasterization within that pixel. So while you do write to multiple samples (like with super sampling), you only execute the fragment shader once, thus saving lots of texture accesses. The same value is written across the samples in the multisample image.

That is still doable with deferred rendering. The geometry pass still makes sense under multisampling rules. Or at least, as much sense as it ever has.

The problem is what you have to do during your lighting pass.

Because your geometry pass was multisampled, your lighting pass must read multisampled data. You cannot do a multisample resolve of the geometry buffers (that would be the "nonsensical" part Wikipedia was talking about); your lighting pass must read each sample and process it. Per light. So if you did an 8x multisample operation, your fragment shader will have to run eight times for each pixel. Per light.

That's generally not a cost that a lot of people are willing to pay.

The alternatives are any of the "I can't believe it's not anti-aliasing" techniques, like FXAA or whatever.

Answer 2

Multi-sample antialiasing samples the visibility of a primitive at several subpixel positions, however the shading is only evaluated once inside a pixel per primitive. Deferred shading samples the geometric attributes, and the shading is deferred into a second phase where the correspondence between primitives and visibility samples is lost, which makes it hard to avoid redundant shading. Furthermore, the geometry buffer can become prohibitively large in case of high screen resolutions and high visibility sampling because each sample needs to store all attributes. ---From Deferred Attribute Interpolation Shading's introduction.

Answer 3

It is absolutely possible to do deferred shading with MSAA, but this will of course add complexity in your renderer.

The naive way to perform this is (I for the moment assume that you apply your lights in a fragment/pixel shader) :

• Render your GBuffers with N samples per pixel (this includes your depth buffer)
• During the light pass, for each pixel, iterate trough the N samples to perform your shading (and "average" the value at the end).

This obviously has a huge cost (since if you run 8 samples per pixel for a full HD render, you are actually shading at 8x Full HD resolution).

Thankfully there are some optimizations that can be applied to this process (I assume that your depth buffer is in a format that supports stencil (d24s8 is common for this):

• Render your GBuffers with N samples per pixel as above.
• Set a stencil bit (if you already use stencil, you need to reserve a bit flag for this), on the stencil state, and set write mask to only write to that bit (I assume that your stencil is cleared to zero beginning of the frame). Disable read test, this bit will be write only.
• Run a "sample edge detection" pass, your want to check between depth/normals/color discrepancies across your samples, if some of them are higher that a certain value, this is an edge and requires per sample shading. If not, shading can be run per pixel. If your pixel is not an edge, perform a discard (so stencil bit set to 1 only contains edge pixels).
• Set your stencil test read mask to the same value as your write mask, and set the comparison to NOT_EQUAL, reference value = 1
• Run your shading pass per pixel (edge pixels will not pass the stencil test, so, fragment shader will not be invoked for those).
• Set your stencil test the comparison to EQUAL
• Run your shading pass per sample pixelxN samples (non edge pixels will not pass the stencil test, so, fragment shader will not be invoked for those).

This costs you 2 more passes, but will allow you to have maybe 5% of your total pixels being run per samples (depending on the scene of course, can be even lower).

Now MSAA and deffered can also work well if you apply your lights in Compute Shader (Tile Based deffered). This is mostly for shadowless point/spot lights. This is a long topic, but roughly you use mini frustum and TGSM to cull lights for a small part of your screen and edge detection, and use TGSM barriers to first apply all non edge Pixels against culled lights, then apply Edge pixels, all in one pass.

Another technique for small lights is to render small clipping quads for all your point lights (you can do a draw instanced of 6 vertices with N instances, where N is the number of lights). Build a quad that encloses your sphere/cone in vertex shader, and perform a point/spot light distance test in fragment shader. This limits the amount of pixels processed by a wide margin. In that case, you use the same stencil technique as above.

At this point you have a non Multisampled Buffer that contains your shaded Opaque content, but you might have some transparent objects (and you want MSAA on those, of course).

So generally transparent objects will go in some form of simplified forward shading process.

Process will be as follows :

• Attach a MSAA color texture (without depth).
• Blit your shaded texture onto the MSAA texture.
• Attach back your depth stencil on top of the MSAA color target.
• Set depth write to false, depth test to either LESS or LESSEQUAL (your preference here).
• Draw transparent objects (which will then have MSAA enabled, and correct depth test).

Notes:

• If you use Compute tile, you can store the per tile light indices/count into a buffer, so transparent pass can take advantage of it (you compute the tile index in your transparent fragment material, then you only need to iterate on precomputed culled lights).
• Since it is now possible to write to buffers as well as textures in fragment shaders (at the same time), if you use the clipping quad technique, you can also build tile data at the same time (compute tile index from the pixel and append the light index in the buffer), so building some acceleration structure for your further transparent pass is also possible with this technique.
• Since you will use stencil for edge detection, you can also reserve another bit that is set by all opaque objects (that will also avoid "sky fragments" to go in the light shaders). It also allows you to set unlit materials (before the draw, just make sure that you do not write into that bit. This will require a blit pass before to apply shading (write all unlit pixels as is in the shaded texture), but this is really fast.