4

I am after smooth texture based outline effect in OpenGL. So far I tried mostly all kinds of edge detection algorithms which result mostly in crude and jagged outlines. Then I read about Distance Field. I found an example which does pretty nice distance field. Here is the GLSL code:

#version 420
layout(binding=0)  uniform sampler2D colorMap;
flat in vec4 diffuseOut;
in vec2 uvsOut;
out vec4 outputColor;
const float ALPHA_THRESHOLD = 0.9;
const float NUM_SPOKES = 36.0; // Number of radiating lines to check in.
const float ANGULAR_STEP =360.0 / NUM_SPOKES;
const int ZERO_VALUE =128; // Color channel containing 0 => -128, 128 => 0, 255 => +127
int in_StepSize=15;    // Distance to check each time (larger steps will be faster, but less accurate).
int in_MaxDistance=30; // Maximum distance to search out to. Cannot be more than 127!

vec4 distField(){

    vec2 pixel_size = 1.0 / vec2(textureSize(colorMap, 0));
    vec2 screenTexCoords = gl_FragCoord.xy * pixel_size;
    int distance;

    if(texture(colorMap, screenTexCoords).a == 0.0)
    {
        // Texel is transparent, search for nearest opaque.
        distance = ZERO_VALUE + 1;
        for(int i = in_StepSize; i < in_MaxDistance; i += in_StepSize)
        {
            if(find_alpha_at_distance(screenTexCoords, float(i) * pixel_size, 1.0))
            {
                i = in_MaxDistance + 1; // BREAK!
            }
            else
            {
                distance = ZERO_VALUE + 1 + i;
            }
        }
    }
    else
    {
        // Texel is opaque, search for nearest transparent.
        distance = ZERO_VALUE;
        for(int i = in_StepSize; i <= in_MaxDistance; i += in_StepSize)
        {
            if(find_alpha_at_distance(screenTexCoords, float(i) * pixel_size, 0.0))
            {
                i = in_MaxDistance + 1; // BREAK!
            }
            else
            {
                distance = ZERO_VALUE - i;
            }
        }
    }

    return  vec4(vec3(float(distance) / 255.0) * diffuseOut.rgb, 1.0 - texture(colorMap, screenTexCoords).a);

}

void main()
{
    outputColor= distField();
}

The result of this shader covers the whole screen using the diffuse color for filling the screen area outside the Distance Field outline.Here is how it looks like : enter image description here

What I need is to leave all the area which has the solid red fill outside the distance field as transparent.

bunnyhero
  • 757
  • 2
  • 10
  • 23
Michael IV
  • 11,016
  • 12
  • 92
  • 223
  • 1
    Do you have any blending enabled? It's not enough to just output an alpha value, you must enable a blend mode that knows how to interpret that alpha as the correct transparency. – Tim Sep 01 '12 at 15:50
  • I agree with @Tim, which probably solves your problem. For completeness though, are you rendering a fullscreen quad? – pauluss86 Sep 01 '12 at 20:14
  • Looking at the provided screenshot carefully, I noticed what might be regarded as artefacts. There appears to be a white line on the inside of the red stroke and the stroke itself appears a bit jagged; especially the 'convex' parts. The first might be solved by making sure the texture coords refer to the center of pixels in the texture map. The second problem doesn't appear in the linked article but might be related to the alphablending problem. – pauluss86 Sep 01 '12 at 20:19

2 Answers2

2

I came to the solution by using Distance Field gray scale 8 bit alpha map.Stefan Gustavson describes in detail how to do it.Basically one needs to generate the distance field version of the original texture.Then this texture is rendered with the primitive normally in the first pass into an FBO.In the second pass the alpha blending mode should be on.The texture from the first pass in used with the screen quad.At this stage the the fragment shader samples the alpha from that texture.This results in both smooth edges and alpha transparency around the edges. Here is the result: enter image description here

Michael IV
  • 11,016
  • 12
  • 92
  • 223
  • Looks very good and I don't see any artefacts anymore. How long did you roughly spend on this implementation? – pauluss86 Sep 03 '12 at 13:13
  • Well , in my case it took a couple of hours as I needed to port the CPU side (Stefan Gustavson's Distance field process methods) from C++ to Java.But if you use C++ by default then you can just copy paste from his examples. :) – Michael IV Sep 03 '12 at 13:17
0

Based on the screenshot I'm assuming you're rendering a fullscreen quad? If that's the case Tim just provided the answer, try:

glEnable( GL_BLEND );
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

Before you render the quad. Obviously if you're going to render non-transparent stuff too, I advise you to render those first so you won't get depth buffer problems. When you're done drawing the transparent stuff, call:

glDisable( GL_BLEND );

To turn alphablending off again.

pauluss86
  • 454
  • 6
  • 9