what's wrong with it - or how to find correct THREE.PerspectiveCamera settings

Question

I have a simple THREE.Scene where the main content is a THREE.Line mesh that visualizes the keyframe based path that the camera will follow for some scripted animation. There is then one THREE.SphereGeometry based mesh that is always repositioned to the current camera location.

The currently WRONG result looks like this (the fractal background is rendered independently but using the same keyframe input - and ultimately the idea is that the "camera path" visualization ends up in the same scale/projection as the respective fractal background...):

The base is an array of keyframes, each of which represents the modelViewMatrix for a specific camera position/orientation and is directly used to drive the vertexshader for the background, e.g.:

  varying vec3 eye, dir;
  void main()   {
    gl_Position = vec4(position, 1.0);
    eye = vec3(modelViewMatrix[3]);
    dir = vec3(modelViewMatrix * vec4(position.x , position.y , 1, 0));
  }

(it is my understanding that "eye" is basically the camera position while "dir" reflects the orientation of the camera and by the way it is used during the ray marching implicitly leads to a perspective projection)

The respective mesh objects are created like this:

visualizeCameraPath: function(scene) {
        // debug: visualize the camera path
        var n= this.getNumberOfKeyFrames();

        var material = new THREE.LineBasicMaterial({
            color: 0xffffff
        });

        var geometry = new THREE.Geometry();
        for (var i= 0; i<n; i++) {
            var m= this.getKeyFrameMatrix(true, i);

            var pos= new THREE.Vector3();
            var q= new THREE.Quaternion();
            var scale= new THREE.Vector3();
            m.decompose(pos,q,scale);

            geometry.vertices.push( new THREE.Vector3( pos.x, pos.y, pos.z ));          
        }

        this.camPath = new THREE.Line( geometry, material );
        this.camPath.frustumCulled = false; // Avoid getting clipped - does not seem to help one little bit
        scene.add( this.camPath );


        var radius= 0.04;
        var g = new THREE.SphereGeometry(radius, 10, 10, 0, Math.PI * 2, 0, Math.PI * 2);

        this.marker = new THREE.Mesh(g, new THREE.MeshNormalMaterial());
        scene.add(this.marker);
    }

in order to play the animation I update the camera and the marker position like this (I guess it is already wrong how I use the input matrix "m" directly on the "shadowCamera" - eventhough I think that it contains the correct position):

syncShadowCamera(m) {
    var pos= new THREE.Vector3();
    var q= new THREE.Quaternion();
    var scale= new THREE.Vector3();
    m.decompose(pos,q,scale);

    this.applyMatrix(m, this.shadowCamera);     // also sets camera position to "pos"


    // highlight current camera-position on the camera-path-line        
    if (this.marker != null) this.marker.position.set(pos.x, pos.y, pos.z);

},

applyMatrix: function(m, targetObj3d) {
    var pos= new THREE.Vector3();
    var q= new THREE.Quaternion();
    var scale= new THREE.Vector3();
    m.decompose(pos,q,scale);

    targetObj3d.position.set(pos.x, pos.y, pos.z);
    targetObj3d.quaternion.set(q.x, q.y, q.z, q.w);
    targetObj3d.scale= scale;

    targetObj3d.updateMatrix(); // this.matrix.compose( this.position, this.quaternion, this.scale );   
    targetObj3d.updateMatrixWorld(true);
},

I've tried multiple things with regard to the camera and the screenshot reflects the output with disabled "this.projectionMatrix" (see below code).

createShadowCamera: function() {        
    var speed = 0.00039507;
    var z_near = Math.abs(speed);
    var z_far = speed * 65535.0;
    var fH = Math.tan( this.FOV_Y * Math.PI / 360.0 ) * z_near;
    var fW = Math.tan( this.FOV_X * Math.PI / 360.0 ) * z_near;
    // orig opengl used: glFrustum(-fW, fW, -fH, fH, z_near, z_far);

    var camera= new THREE.PerspectiveCamera();          
    camera.updateProjectionMatrix =  function() {
    //  this.projectionMatrix.makePerspective( -fW, fW, fH, -fH, z_near, z_far );
        this.projectionMatrix= new THREE.Matrix4();     // hack: fallback to no projection
    };
    camera.updateProjectionMatrix();
    return camera;
},

My initial attempt had been to use the same kind of settings that the opengl shader for the fractal background had been using (see glFrustum above). Unfortunately it seems that I have yet managed to correctly map the input "modelViewMatrix" (and the projection implicitly performed by the raymarching in the shader) to equivalent THREE.PerspectiveCamera settings (orientation/projectionMatrix).

Is there any matrix calculation expert here, that knows how to obtain the correct transformations?

wothke · Accepted Answer · 2017-03-13T11:47:36.800

Finally I have found one hack that works.

Actually the problem was made up of two parts:

1) Row- vs column-major order of modelViewMatrix: The order expected by the vertex shader is the oposite of what the remaining THREE.js expects..

2) Object3D-hierarchy: i.e. Scene, Mesh, Geometry, Line vertices + Camera: where to put the modelViewMatrix data so that it creates the desired result (i.e. the same result that the old bloody opengl application produced): I am not happy with the hack that I found here - but so far it is the only one that seems to work:

I DO NOT touch the Camera.. it stays at 0/0/0
I directly move all the vertices of my "line"-Geometry relative to the real camera position (see "position" from the modelViewMatrix)
I then disable "matrixAutoUpdate" on the Mesh that contains my "line" Geometry and copy the modelViewMatrix (in which I first zeroed out the "position") into the "matrix" field.

BINGO.. then it works. (All of my attempts to achieve the same result by rotating/displacing the Camera or by displacing/rotating any of the Object3Ds involved have failed miserably..)

EDIT: I found a better way than updating the vertices and at least keeping all the manipulations on the Mesh level (I am still moving the world around - like the old OpenGL app would have done..). To get the right sequence of translation/rotation one can also use ("m" is still the original OpenGL modelViewMatrix - with 0/0/0 position info):

var t= new THREE.Matrix4().makeTranslation(-cameraPos.x, -cameraPos.y, -cameraPos.z);
t.premultiply ( m );

obj3d.matrixAutoUpdate=false;
obj3d.matrix.copy(t);

If somebody knows a better way that also works (one where the Camera is updated without having to directly manipulate object matrices) I'd certainly be interested to hear it.

what's wrong with it - or how to find correct THREE.PerspectiveCamera settings

1 Answers1