What is this technique, if not bilinear filtering?

Question

I'm trying to replicate the automatic bilinear filtering algorithm of Unity3D using the next code:

fixed4 GetBilinearFilteredColor(float2 texcoord)
{
    fixed4 s1 = SampleSpriteTexture(texcoord + float2(0.0, _MainTex_TexelSize.y));
    fixed4 s2 = SampleSpriteTexture(texcoord + float2(_MainTex_TexelSize.x, 0.0));
    fixed4 s3 = SampleSpriteTexture(texcoord + float2(_MainTex_TexelSize.x, _MainTex_TexelSize.y));
    fixed4 s4 = SampleSpriteTexture(texcoord);

    float2 TexturePosition = float2(texcoord)* _MainTex_TexelSize.z;

    float fu = frac(TexturePosition.x);
    float fv = frac(TexturePosition.y);

    float4 tmp1 = lerp(s4, s2, fu);
    float4 tmp2 = lerp(s1, s3, fu);

    return lerp(tmp1, tmp2, fv);
}

fixed4 frag(v2f IN) : SV_Target
{
    fixed4 c = GetBilinearFilteredColor(IN.texcoord) * IN.color;
    c.rgb *= c.a;
    return c;
}

I thought I was using the correct algoritm because is the only one I have seen out there for bilinear. But I tried it using unity with the same texture duplicated:

1º texture: is in Point filtering and using the custom bilinear shader (maded from the default sprite shader).
2º texture: is in Bilinear filter with the default sprite shader

And this is the result:

You can see that they are different and also there is some displacement in my custom shader that makes the sprite being off-center when rotating in the Z axis.

Any idea of what I'm doing wrong? Any idea of what is doing Unity3D different? Are there another algorithm's who fits in the Unity3D default filtering?

Solution

Updated with the complete code solution with Nico's code for other people who search for it here:

fixed4 GetBilinearFilteredColor(float2 texcoord)
{
    fixed4 s1 = SampleSpriteTexture(texcoord + float2(0.0, _MainTex_TexelSize.y));
    fixed4 s2 = SampleSpriteTexture(texcoord + float2(_MainTex_TexelSize.x, 0.0));
    fixed4 s3 = SampleSpriteTexture(texcoord + float2(_MainTex_TexelSize.x, _MainTex_TexelSize.y));
    fixed4 s4 = SampleSpriteTexture(texcoord);

    float2 TexturePosition = float2(texcoord)* _MainTex_TexelSize.z;

    float fu = frac(TexturePosition.x);
    float fv = frac(TexturePosition.y);

    float4 tmp1 = lerp(s4, s2, fu);
    float4 tmp2 = lerp(s1, s3, fu);

    return lerp(tmp1, tmp2, fv);
}

fixed4 frag(v2f IN) : SV_Target
{
    fixed4 c = GetBilinearFilteredColor(IN.texcoord - 0.498 * _MainTex_TexelSize.xy) * IN.color;
    c.rgb *= c.a;
    return c;
}

And the image test with the result:

Why don't substract 0.5 exactly?

If you test it you will see some edge cases where it jumps to (pixel - 1).

score 9 · Accepted Answer · answered Sep 08 '16 at 00:31

Let's take a closer look at what you are actually doing. I will stick to the 1D case because it is easier to visualize.

You have an array of pixels and a texture position. I assume, _MainTex_TexelSize.z is set in a way, such that it gives pixel coordinates. This is what you get (the boxes represent pixels, numbers in boxes the pixel number and numbers below the pixel space coordinates):

With your sampling (assuming nearest point sampling), you will get pixels 2 and 3. However, you see that the interpolation coordinate for lerp is actually wrong. You will pass the fractional part of the texture position (i.e. 0.8) but it should be 0.3 (= 0.8 - 0.5). The reasoning behind this is quite simple: If you land at the center of a pixel, you want to use the pixel value. If you land right in the middle between two pixels, you want to use the average of both pixel values (i.e. an interpolation value of 0.5). Right now, you have basically an offset by a half pixel to the left.

When you solve the first problem, there is a second one:

In this case, you actually want to blend between pixel 1 and 2. But because you always go to the right in your sampling, you will blend between 2 and 3. Again, with a wrong interpolation value.

The solution should be quite simple: Subtract half of the pixel width from the texture coordinate before doing anything with it, which is probably just the following (assuming that your variables hold the things I think):

fixed4 c = GetBilinearFilteredColor(IN.texcoord - 0.5 * _MainTex_TexelSize.xy) * IN.color;

Another reason why the results are different could be that Unity actually uses a different filter, e.g. bicubic (but I don't know). Also, the usage of mipmaps could influence the result.

What is this technique, if not bilinear filtering?

1 Answers1