three.js: Get updated vertices with skeletal animations?

Question

Similar to the question in this stack overflow question Three.js: Get updated vertices with morph targets I am interested in how to get the "actual" position of the vertices of a mesh with a skeletal animation.

I have tried printing out the position values, but they are never actually updated (as I am to understand, this is because they are calculated on the GPU, not CPU). The answer to the question above mentioned doing the same computations on the CPU as on the GPU to get the up to date vertex positions for morph target animations, but is there a way to do this same approach for skeletal animations? If so, how??

Also, for the morph targets, someone pointed out that this code is already present in the Mesh.raycast function (https://github.com/mrdoob/three.js/blob/master/src/objects/Mesh.js#L115 ). However, I don't see HOW the raycast works with skeletal animation meshes-- how does it know the updated position of the faces?

Thank you!

Answer 1

A similar topic was discussed in the three.js forum some time ago. I've presented there a fiddle which computes the AABB for a skinned mesh per frame. The code actually performs the same vertex displacement via JavaScript like in the vertex shader. The routine looks like so:

function updateAABB( skinnedMesh, aabb ) {

    var skeleton = skinnedMesh.skeleton;
    var boneMatrices = skeleton.boneMatrices;
    var geometry = skinnedMesh.geometry;

    var index = geometry.index;
    var position = geometry.attributes.position;
    var skinIndex = geometry.attributes.skinIndex;
    var skinWeigth = geometry.attributes.skinWeight;

    var bindMatrix = skinnedMesh.bindMatrix;
    var bindMatrixInverse = skinnedMesh.bindMatrixInverse;

    var i, j, si, sw;

    aabb.makeEmpty();

    // 

    if ( index !== null ) {

        // indexed geometry

        for ( i = 0; i < index.count; i ++ ) {

            vertex.fromBufferAttribute( position, index[ i ] );
            skinIndices.fromBufferAttribute( skinIndex, index[ i ] );
            skinWeights.fromBufferAttribute( skinWeigth, index[ i ] );

            // the following code section is normally implemented in the vertex shader

            vertex.applyMatrix4( bindMatrix ); // transform to bind space
            skinned.set( 0, 0, 0 );

            for ( j = 0; j < 4; j ++ ) {

                si = skinIndices.getComponent( j );
                sw = skinWeights.getComponent( j );
                boneMatrix.fromArray( boneMatrices, si * 16 );

                // weighted vertex transformation

                temp.copy( vertex ).applyMatrix4( boneMatrix ).multiplyScalar( sw );
                skinned.add( temp );

            }

            skinned.applyMatrix4( bindMatrixInverse ); // back to local space

            // expand aabb

            aabb.expandByPoint( skinned );

        }

    } else {

        // non-indexed geometry

        for ( i = 0; i < position.count; i ++ ) {

            vertex.fromBufferAttribute( position, i );
            skinIndices.fromBufferAttribute( skinIndex, i );
            skinWeights.fromBufferAttribute( skinWeigth, i );

            // the following code section is normally implemented in the vertex shader

            vertex.applyMatrix4( bindMatrix ); // transform to bind space
            skinned.set( 0, 0, 0 );

            for ( j = 0; j < 4; j ++ ) {

                si = skinIndices.getComponent( j );
                sw = skinWeights.getComponent( j );
                boneMatrix.fromArray( boneMatrices, si * 16 );

                // weighted vertex transformation

                temp.copy( vertex ).applyMatrix4( boneMatrix ).multiplyScalar( sw );
                skinned.add( temp );

            }

            skinned.applyMatrix4( bindMatrixInverse ); // back to local space

            // expand aabb

            aabb.expandByPoint( skinned );

        }

    }

    aabb.applyMatrix4( skinnedMesh.matrixWorld );

}

Also, for the morph targets, someone pointed out that this code is already present in the Mesh.raycast function

Yes, you can raycast against morphed meshes. Raycasting against skinned meshes is not supported yet. The code in Mesh.raycast() is already very complex. I think it needs some serious refactoring before it is further enhanced. In the meantime, you can use the presented code snippet to build a solution by yourself. The vertex displacement logic is actually the most complicated part.

Live demo: https://jsfiddle.net/fnjkeg9x/1/

three.js R107