Custom HLSL shader making weird patterns across icosphere

Question

really hoping that someone can help me here - I rarely can't resolve bugs in C# since I have a fair amount of experience in it but I don't have a lot to go on with HLSL.

The picture linked to below is of the same model (programmatically generated on run) twice, the first (white) time using BasicEffect and the second time using my custom shader, listed below. The fact that it works with BasicEffect makes me think that it's not an issue with generating the normals for the model or anything like that.

I've included different levels of subdividing to better illustrate the issue. It's worth mentioning that both effects are using the same lighting direction.

https://imagizer.imageshack.us/v2/801x721q90/673/qvXyBk.png

Here's my shader code (feel free to pick it apart, any tips are very welcome):

float4x4 WorldViewProj;

float4x4 NormalRotation = float4x4(1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1);

float4 ModelColor = float4(1, 1, 1, 1);
bool TextureEnabled = false;
Texture ModelTexture;
sampler ColoredTextureSampler = sampler_state
{
    texture = <ModelTexture>;
    magfilter = LINEAR; minfilter = LINEAR; mipfilter = LINEAR;
    AddressU = mirror; AddressV = mirror;
};

float4 AmbientColor = float4(1, 1, 1, 1);
float AmbientIntensity = 0.1;
float3 DiffuseLightDirection = float3(1, 0, 0);
float4 DiffuseColor = float4(1, 1, 1, 1);
float DiffuseIntensity = 1.0;

struct VertexShaderInput
{
    float4 Position : POSITION0;
    float4 Normal : NORMAL0;
    float2 TextureCoordinates : TEXCOORD0;
};

struct VertexShaderOutput
{
    float4 Position : POSITION0;
    float4 Color : COLOR0;
    float2 TextureCoordinates : TEXCOORD0;
};

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
{
    VertexShaderOutput output = (VertexShaderOutput)0;

    output.Position = mul(input.Position, WorldViewProj);

    float4 normal = mul(input.Normal, NormalRotation);
    float lightIntensity = dot(normal, DiffuseLightDirection);
    output.Color = saturate(DiffuseColor * DiffuseIntensity * lightIntensity);

    output.TextureCoordinates = input.TextureCoordinates;

    return output;
}

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
    float4 pixBaseColor = ModelColor;
    if (TextureEnabled == true)
    {
        pixBaseColor = tex2D(ColoredTextureSampler, input.TextureCoordinates);
    }
    float4 lighting = saturate((input.Color + AmbientColor * AmbientIntensity) * pixBaseColor);
    return lighting;
}

technique BestCurrent
{
    pass Pass1
    {
        VertexShader = compile vs_2_0 VertexShaderFunction();
        PixelShader = compile ps_2_0 PixelShaderFunction();
    }
}

A couple things to try: 1) Make sure the normal is normalized before using it in the dot product. 2) The normal and light direction vector need to point in the same direction for the dot product (out from the surface). If your intent for light is +x direction, then you need to negate it to -x direction before doing the dot product. 3) Probably not an issue, but ensure either the fourth component of `input.Normal` is 0 or the `NormalRotation` matrix contains no translations. — megadan, Oct 08 '14 at 06:40
Also make sure the light direction is normalized before using it in the dot product — megadan, Oct 08 '14 at 15:14
That first sentence solved my problem right away with normalising the normals. Could have spent days overlooking that! Thanks very much for the quick reply. If you send that as an answer, I'll mark it. — Phil Ekiert, Oct 08 '14 at 21:53

score 0 · Accepted Answer · answered Oct 08 '14 at 23:31

In general, when implementing a lighting equation, there are a few things to ensure:

Normals, light directions, and other directional vectors should be normalized before using them in a dot product. In your case you could add something like:

normal = normalize(normal);

The same should be done for DiffuseLightDirection if it is already not normalized. It already is with your default value, but if your app changes it, it might not be normalized anymore. For that, it would be better to normalize in the application code since it only needs to be done once when it changes, and not per vertex.

Also remember that if you are multiplying the vector by a matrix that contains a scale, the vector will no longer be normalized, so it will need to be re-normalized.
The light direction and the normal must point in the same direction which is out from the surface. Your default light direction is (1,0,0). If you want light to point in the +x direction, then you must actually negate the vector before performing the dot product with the normal so that it is pointing out from the surface just like the normal. If you already take this into account, then it's not a problem.
Vectors can't be translated since they are just a direction not a position. So it is important to ensure when you transform them with a matrix that either the fourth component (w) of the vector is 0 or the matrix you are transforming it with has no translation. Setting w to 0 will zero out any translation from the matrix during the multiply. Since your matrix is called NormalRotation, I'm assuming it only contains a rotation, so this probably isn't an issue.

Custom HLSL shader making weird patterns across icosphere

1 Answers1