1

I have a batched rendering pipeline that uploads a bunch of quads to OpenGL. These are rendered to an FBO. Naturally, one of the attachments is visual output and another is depth, allowing for post-processing etc. However, a third attachment is intended for storing object IDs for picking, in a similar style to that explained here.

There's a small catch. I would like some rendered items to be 'transparent' to picking. Essentially, some things (particles or visual decoration or whatever) are not pickable themselves, but should not prevent the picking of objects behind them.

How might this be accomplished? Web searches suggest that glAlphaFunc might once have fulfilled this role, as I could have packed the object id into a texture with a transparency channel, and then deleted any fragments which I set the transparency to 0. A bit fiddly, but workable. However, that's been depreciated.

The other common recommendation for similar problems is to use the discard keyword. This isn't workable because it deletes all output elements of the fragment, not just for a specified attachment.

What's the solution here? Is there something clever I can do with blend functions that I just can't see? I'm pretty flummoxed.

Fragment shader looks something like this:

#region Fragment

#version 440 core

layout(location = 0) out vec4 colour; // Visual
layout(location = 1) out uint interactLayer; // Picking

in vec2 texCoord;
in flat int texUnit;
in flat uint objectID;

uniform sampler2D[32] u_texData;

void main()
{
    vec4 c1 = fragmentColour * texture(u_texData[texUnit], texCoord);
    
    if (c1.w == 0.0f) { discard; }
    colour = c1;
    
    if (objectID == 0xffffffffu) {
        // Please don't write to interactLayer values,
        // but also please don't discard the 'colour' value.
    }
    else {
        interactLayer = objectID;
    }
}
Stefan Bauer
  • 393
  • 1
  • 9
  • 1
    "*This isn't workable because it deletes all output elements of the fragment, not just for a specified attachment.*" That would have happened with alpha func too. – Nicol Bolas Sep 02 '20 at 00:59
  • What you want to do here can only work if you manually depth-sort your scene. – derhass Sep 02 '20 at 01:15
  • @NicolBolas That's a good point; I suppose in this hypothetical scenario I was assuming the modern inclusion of a `glAlphaFunci`, in the same way that there's both `glBlendFunc` and `glBlendFunci`. In any case, it's not to be. – Stefan Bauer Sep 02 '20 at 01:51
  • @derhass I am indeed sorting my quads before sending them to the shader. The implication would seem to be that there is indeed a solution; if so, any pointers would be greatly appreciated. – Stefan Bauer Sep 02 '20 at 01:53

1 Answers1

2

Discarding a fragment is binary; it is either discarded (along with all its results) or it is accepted.

You also can't do blending to a non-normalized integer image format. Blending only works between types that are floating point (including normalized integers).

Your best bet is to just use a float image format for your index. Cast your integer to the shader, then employ blending to turn it on/off. This will require writing a vec4 in the shader so that you can write an appropriate alpha value. Note that just because you're writing a vec4 doesn't mean the image format must have 4 components; it can still have 1 component and blending ought to work (as long as you're only using the source alpha).

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
  • Thanks for the help! Is there any chance you could expand further on that last sentence? I'm not entirely sure what you mean or what that would look like in code. – Stefan Bauer Sep 02 '20 at 12:42