Outer artifacts when using GL_LINEAR in Texture OpenGL

Question

I success to generate the height map using texture in openGL. The full picture of this data is shown in the figure below. However, when I'm using GL_LINEAR option in my glTexParameteri (MinMag) it give me outer artifacts.

this is my glTexParameter

if (interpolate) {
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
}
else {
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
}

glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

Here is my fragment shader code

    "varying vec2 tex;\n"
    "uniform int colorSizeI;\n"
    "uniform vec4 colorTable[512];\n"
    "uniform vec2 minmaxZ;\n"
    "uniform vec3 backgroundColor;\n"
    "uniform sampler2D diffuse;\n"
    "uniform float transparency;\n"
    "void main() {\n"
    "   float color = texture2D(diffuse, tex).r;\n"
    "   float h = (color - minmaxZ[0])/(minmaxZ[1] - minmaxZ[0]);\n"
    "   float colorSizeF = float(colorSizeI);\n"
    "   int i = 0;\n"
    "   float j = 0.0f;\n"
    "   vec3 base;\n"
    "   if (color == 0.0f) {\n"
    "       base = vec3(0.0f, 0.0f, 0.0f);}\n"
    "   else if (color <= minmaxZ[0]) {\n"
    "       base = colorTable[0].xyz;}\n"
    "   else if (color >= minmaxZ[1]) {\n"
    "       base = colorTable[colorSizeI - 1].xyz;}\n"
    "   else { \n"
    "       while (h >= (j + 1.0f)/colorSizeF) {\n"
    "           i += 1;\n"
    "           j += 1.0f;}\n"
    "       base = mix(colorTable[i].xyz, colorTable[i+1].xyz, colorSizeF*(h - (j/colorSizeF)));}\n"
    "   gl_FragColor = vec4(base, transparency);\n"
    "}\n";

Here is the image when using GL_Nearest (everything is OK)

Here is the image when using GL_Linear (outer interpolated artifacts comes out)

So, anyone know how to remove this artifacts ?

NB: The outer black color is consider the data also when color = 0.

I used GL_CLAMP_TO_EDGE, I forgot to mention, the outer black color outside the square actually data also when the z = 0; So I don't believe it's because the texturewrap. — zufryy, Jan 24 '18 at 16:06
you're going to have to provide all the `::glTexParameteri` calls so we can see what you're doing — keith, Jan 24 '18 at 16:11
what happens if you do `gl_FragColor = texture2D(diffuse, tex)` — keith, Jan 24 '18 at 16:13
I'm using texture to avoid using the vertices & indices (reduces memory). In the texture (red) data is contain the heightmap that will be show in color based on the calculation in fragment shader. So if I do `gl_FragColor = texture2D(diffuse, tex)` it won't display the data that I want to show. — zufryy, Jan 24 '18 at 16:22

score 2 · Answer 1 · answered Jan 24 '18 at 16:00

2

I assume your wrapping for the texture is currently GL_REPEAT. You should try another one.

GL_MIRRORED_REPEAT or GL_CLAMP_TO_EDGE should work.

answered Jan 24 '18 at 16:00

Honolorakahl

21
1

I used GL_CLAMP_TO_EDGE, I forgot to mention, the outer black color outside the square actually data also when the z = 0; So I don't believe it's because the texturewrap. – zufryy Jan 24 '18 at 16:06
Ok, then maybe this line is the problem: base = colorTable[colorSizeI - 1].xyz;} Looks wrong but can't help you more because I don't know the uniform values. – Honolorakahl Jan 24 '18 at 16:24
I prepare buffer in `uniform vec4 colorTable[512]` because have some option for my colorTable and every option of that has different number of color. The `colorSizeI - 1` is to make sure if the height more than some value, color will be the last valid value in colorTable. So I tried to change it like you suggest, but the interpolation artifacts still there. – zufryy Jan 24 '18 at 16:34
I think now I understand what you're trying to do: you have a texture with values from 0.0 to 1.0 for height but this texture has a border with value 1.0 = maximum height. OpenGL is now interpolating these values in the middle along the border with 1.0. - Looks like you have to remove the border from the texture or stay with GL_NEAREST. – Honolorakahl Jan 24 '18 at 16:49

Andreas · Answer 2 · 2018-01-24T20:03:48.733

1

The error is because of interpolating from "null color" to actual colors in the colr table. To put it short: you can't do that. The concept is broken. Back to the drawing board.

The simplest solution is not having null color, limiting the quad to only the valid area, and doing GL_CLAMP_TO_EDGE as others suggested.

A trickier way is setting GL_NEAREST and then writing the linear sampling yourself in the shader, sampling the texture with tex plus some offsets (manhattan distance should be fine) and skip the null color when blending (the skip part is what is missing from your original concept). The tricky part is getting the offsets right.

edited Jan 24 '18 at 20:03

answered Jan 24 '18 at 19:58

Andreas

5,086
3
16
36

Could use "transparent (alpha = 0.0)" as the null color, and then the predefined blend functions should work again. – Ben Voigt Jan 24 '18 at 20:06
Sadly not. It would at best remove half a texelwidth of artifacts but the null color would still bleed into non-null texels, e.g. top of top-left texel would turn from red to yellow whereas zufry wants it to blue to the border. – Andreas Jan 24 '18 at 20:13
For the area with black color, I will put alpha = 0 there. I show it in black color for make thing easier to explain here. And also the area with black color it has height = 0 and it goes into these `if (color == 0.0f) {base = vec3(0.0f, 0.0f, 0.0f);}`. @Andreas: so in the end it's a limitation of GL_LINEAR ? can you give reading material about method your suggest ? – zufryy Jan 24 '18 at 21:10
@zufryy It is not a method I know of from before. If you want to read something check the documentation for GL_LINEAR and GL_NEAREST. Make sure you understand why you get the erroneous image output, and try figure out an algorithm that satisfies youy requirements from there. Good luck. – Andreas Jan 27 '18 at 13:08
@Andreas: The things is when using GL_LINEAR when the pixel try to calculate interpolation and there is null point in the neighbour pixels it will have this kind of situation. Thus, I just make "dummy" point as an outer boundaries then apply alpha = 0 there. – zufryy Jan 27 '18 at 15:13

zufryy · Accepted Answer · 2018-01-27T15:08:47.870

1

I managed to fixed it with making an outer boundary with the value of interpolation of nearest pixels and put alpha = 0 for that.

It worked nicely, however it cost 2 times of original memory consumption since I need to store the alpha map also.

edited Jan 27 '18 at 15:08

answered Jan 26 '18 at 23:45

zufryy

177
2
10

please post a picture of the final result – Andreas Jan 27 '18 at 12:53
little bit blurry on the edge. But I can accept it. – zufryy Jan 27 '18 at 15:09

Outer artifacts when using GL_LINEAR in Texture OpenGL

3 Answers3