5

Im currently working on a OpenGL planet rendering. I'm using the Tessellation pipeline. So far things are going very well bar one issue. It's at the stage where I've been banging my head off it for ages and feel like progress isnt happening.

First of all here is a gif of what I'm dealing with. Essentially my problem is that whenever the mouse is moved the planet rotates as if its "looking" at where the camera is pointing.

There are some graphical issues but they are due to me simply repeating the same heightmap across the whole cubemap. Since it doesnt match up on the sides there are clear seams.

Below is my evaluation shader:

void main(void) {
    vec4 p0 = gl_in[0].gl_Position;
    vec4 p1 = gl_in[1].gl_Position;
    vec4 p2 = gl_in[2].gl_Position;
    vec3 p = gl_TessCoord.xyz;

    Normal_FS_in = mat3(transpose(inverse((MV)))) * (p0*p.x + (p1*p.y) + p2*p.z).xyz;

    float displacment = texture(tex1, Normal_FS_in).r * 800;
    gl_Position = MVP*(p0*p.x + (p1*p.y) + p2*p.z) + (vec4(vec3(displacment,displacment,0) * normalize(Normal_FS_in),1));
}

Its pretty simple. The shader calculates a normal for the output vertex. This is then used to grab a displacement value from the heightmap which is a cube map. Then GL_Position is calculated. What I've been trying to work out is if my problem lies in the shaders or in the rest of the package.

My attempts have largely been tinkering. I've moved all normal related stuff into the evaluation shader rather then calculating it in the vertex shader and passing them through to the control and evaluation shaders.

The issue only occurs when the mouse is moved.

Any suggestions would be fantastic - hopefully all I'll need is a pointed in the right direction.

Edit - Doing the inverse transpose of the modelview matrix OR view matrix yields the result in the gif. Using the model matrix leads to distortion of the normals when the camera is moved, such that the terrain bends about the place.

Brian
  • 77
  • 2
  • Just so you know, you _might_ get a better answer in http://gamedev.stackexchange.com . It's on topic here too though. – chbaker0 Mar 17 '15 at 05:10
  • Thanks for the suggestion - ill give it a shot over there. – Brian Mar 17 '15 at 05:15
  • If you decide to post there, read http://meta.stackexchange.com/questions/64068/is-cross-posting-a-question-on-multiple-stack-exchange-sites-permitted-if-the-qu. Also: `(p0*p.x + (p1*p.y) + p2*p.z).xyz` this looks funky...are you trying to interpolate the position of the tessellated vertex? Because I'm not sure if that'll accomplish what you want. – chbaker0 Mar 17 '15 at 05:17
  • Yep, thats the goal. Based on what I can see it appears to be doing it correctly. Using that when no displacement is applied leads to the sphere being drawn without any problems. Also thanks for the link, ill bare it in mind. – Brian Mar 17 '15 at 05:20
  • `(vec4(vec3(displacment,displacment,0) * normalize(Normal_FS_in),1))` also seems weird to me...I could well be wrong but shouldn't it be `normalize(Normal_FS_in) * displacement`, since it appears you are doing the displacement in world space? Also, if you are doing the displacement in normal space you should multiply your projection matrix after adding your position and displacement vector. – chbaker0 Mar 17 '15 at 05:33
  • I cant say in 100% certain, but I think its working correctly. The displacement of the terrain shown in the gif is what I'd expect with the heightmap that I supplied. I think the problem lies in the view matrix, simply because the strange planet rotation only occurs when the mouse is moved. – Brian Mar 17 '15 at 05:48
  • Quick update, just in case your interested. Appears that the vertices are in a constant position, but the actual texture related stuff is moving. So in all likelyhood something view matrix related. – Brian Mar 17 '15 at 06:06
  • I see you are indexing into your texture with your normal vector...that doesn't seem right to me. Wouldn't you want your displacement to vary with texture coordinates? – chbaker0 Mar 18 '15 at 01:51
  • may be silly but it looks like simple fisheye error to me. try to change znear and projection angle of your Camera FRUSTRUM ... this is mostly happening due to: 1. too big projection angle 2. object is too near to znear plane so to be sure move the planet a bit farrer from camera and if the effect is smaller then you got your answer – Spektre Apr 15 '15 at 14:32
  • I think you should investigate the MVP. You can eliminate the displacement for simplification (also if there is no scaling in MV, then transpose(inverse((MV)) does nothing because it will just do the inverse twice). - let me know how it goes – Sulea Cosmin Apr 19 '15 at 10:37

2 Answers2

0

It could be a number of things. Here are some examples: 1) You messed up something in the shader (forgot an inverse or transform) 2) You hooked up the controls wrong (mouse left does something strange). 3) The control model is messed up (you move the target rather than the camera). 4) Something else. 5) Some/All of the above.

Try debugging a bit. Does it look ok if you disable the shader completely? Just draw a simple cube (might be easier to track than a sphere).

If it's ok, then add the shader but don't do anything in it, see if the transforms in the shader are reasonable (is it still a cube).

This should help narrow down on possible causes.

Sorin
  • 11,863
  • 22
  • 26
0

Problem is this:

gl_Position = MVP*(p0*p.x + (p1*p.y) + p2*p.z) + (vec4(vec3(displacment,displacment,0) * normalize(Normal_FS_in),1));

You project some position and AFTER that add screen-space displacement. Hard to say what exactly is wrong without seeing whole shader. But you probably missed some parentheses.

Also you are mixing projected vector without perspective divison.

MVP*(p0*p.x + (p1*p.y) + p2*p.z)

w part of this vector is not 1, so your displacement will be scaled based to camera position

Neithy
  • 139
  • 4