I've found a number of posts stating that the fog fragment distance can easily be obtained using gl_Position.z or gl_FragCoord.z/gl_FragCoord.w.
The problem is this makes a boundary where the fog begins.
What I want to do is determine this distance based on player's position such that there is a radius of fog around the player.
For simplicity say I want fog to be interpolated based on a pixel's distance from (0, 0, 0) in world space. Only pixels within fog_end of this coordinate are visible.
You can interpolate the fog at the boundary.
//distance of fragment in worldspace
float distance = ...;
float fog_start = 10;
float fog_end = 20;
//linear interpolation
float fog_factor = (distance-fog_start)/(fog_end-fog_start);
fog_factor = clamp(fog_factor,0,1);
//add 'white fog' to color
out_color += vec4(fog_factor);
The worldspace distance can be obtained by passing the vertex in viewspace to the fragment shader and calculating the length.
After some more searching I found what I am referring to is called range based fog rather than plane based fog.
I also had a problem with having to few vertexes for this to work properly. As such the distance calculation needs to be in the fragment shader.
Some helpful details can be found here.
http://web.archive.org/web/20160615092925/http://in2gpu.com/2014/07/22/create-fog-shader/
