Kinect skeleton Scaling strange behaviour

Question

I am trying to scale a skeleton to match to the sizes of another skeleton. My algoritm do the following:

Find the distance between two joints of the origin skeleton and the destiny skeleton using phytagorean teorem
divide this two distances to find a multiply factor.
Multiply each joint by this factor.

Here is my actual code:

public static Skeleton ScaleToMatch(this Skeleton skToBeScaled, Skeleton skDestiny)
    {
        Joint newJoint = new Joint();

        double distanciaOrigem = 0;
        double distanciaDestino = 0;
        double fator = 1;
        SkeletonPoint pos = new SkeletonPoint();

        foreach (BoneOrientation bo in skToBeScaled.BoneOrientations)
        {
            distanciaOrigem = FisioKinectCalcs.Distance3DBetweenJoint(skToBeScaled.Joints[bo.StartJoint], skToBeScaled.Joints[bo.EndJoint]);
            distanciaDestino = FisioKinectCalcs.Distance3DBetweenJoint(skDestiny.Joints[bo.StartJoint], skDestiny.Joints[bo.EndJoint]);

            if (distanciaOrigem > 0 && distanciaDestino > 0)
            {
                fator = (distanciaDestino / distanciaOrigem);

                newJoint = skToBeScaled.Joints[bo.EndJoint]; // escaling only the end joint as the BoneOrientatios starts from HipCenter, i am scaling from center to edges.

                // applying the new values to the joint
                pos = new SkeletonPoint()
                {
                    X = (float)(newJoint.Position.X * fator),
                    Y = (float)(newJoint.Position.Y * fator),
                    Z = (float)(newJoint.Position.Z * fator)
                };

                newJoint.Position = pos;
                skToBeScaled.Joints[bo.EndJoint] = newJoint;
            }
        }

        return skToBeScaled;
    }

Every seems to work fine except for the hands and foots

Look at this images

enter image description here

I have my own skeleton over me, and my skeleton scaled to the sizes of another person, but the hands and foots still crazy. (but code looks right)

Any suggestion?

Just realised the age of this ... very late to the party! I think your problem stems from trying to scale you skeleton on the fly while still using it to compute original distances. Create a third scaled skeleton that you can return at the end. Do not modify the original joint position in the loop since you need them intact for correct distance calculations. — zeFrenchy, Aug 14 '17 at 09:01

score 1 · Answer 1 · answered Nov 28 '12 at 23:13

1

It's hard to say without running the code, but it somewhat "looks good".

What I would validate though, is your

if (distanciaOrigem > 0 && distanciaDestino > 0)

If distanciaOrigem is very close to 0, but even just epsilon away from 0, it won't be picked up by the if, and then

fator = (distanciaDestino / distanciaOrigem);

Will result in a very large number!

answered Nov 28 '12 at 23:13

emartel

7,712
1
30
58

Thanks for your answer. Seems good, will try it later and give you the feedback. – Ewerton Nov 28 '12 at 23:17
I am brazilian with a bad english, sorry, whats epsilon, some fraction? – Ewerton Nov 28 '12 at 23:17
@Ewerton epsilon can be used to mean a very small number - see this thread http://stackoverflow.com/questions/2411392/double-epsilon-for-equality-greater-than-less-than-less-than-or-equal-to-gre – emartel Nov 28 '12 at 23:19
@Ewerton to take errors into account, you might even want to skip the last bones (hands and feet) – emartel Nov 28 '12 at 23:21
but i need to scale these joints too. – Ewerton Nov 28 '12 at 23:23
@Ewerton I would be surprised if it made much of a difference but anyway, is your `fator` a very big number because `distanciaOrigem` is very small? – emartel Nov 28 '12 at 23:26
seems that it is the problem, as i am travesing the BoneOrientations colletion, the first pair is (HipCenter, HipCenter) that is (0, 0), so i do the if to avoid division by zero. But accordingly you, my if needs to be if (distanciaOrigem == 0 && distanciaDestino == 0) i will change the code and test it later. – Ewerton Nov 28 '12 at 23:30
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/20350/discussion-between-emartel-and-ewerton) – emartel Nov 29 '12 at 20:38

ZimaXXX · Answer 2 · 2014-02-06T16:03:17.173

I would suggest to smooth the factor so it generally fits the proper scale. Try this code:

    private static Dictionary<JointType, double> jointFactors = null;
    static CalibrationUtils()
    {
        InitJointFactors();
    }
    public static class EnumUtil
    {
        public static IEnumerable<T> GetValues<T>()
        {
            return Enum.GetValues(typeof(T)).Cast<T>();
        }
    }
    private static void InitJointFactors()
    {
        var jointTypes = EnumUtil.GetValues<JointType>();
        jointFactors = new Dictionary<JointType, double>();
        foreach(JointType type in jointTypes)
        {
            jointFactors.Add(type, 0);
        }
    }
    private static double SmoothenFactor(JointType jointType, double factor, int weight)
    {
        double currentValue = jointFactors[jointType];
        double newValue = 0;
        if(currentValue != 0)
            newValue = (weight * currentValue + factor) / (weight + 1);
        else
            newValue = factor;
        jointFactors[jointType] = newValue;
        return newValue;
    }

When it comes to factor usage just use the SmoothenFactor method first:

    public static Skeleton ScaleToMatch(this Skeleton skToBeScaled, Skeleton skDestiny, double additionalFactor = 1)
    {
        Joint newJoint = new Joint();

        double distanceToScale = 0;
        double distanceDestiny = 0;
        double factor = 1;
        int weight = 500;
        SkeletonPoint pos = new SkeletonPoint();
        Skeleton newSkeleton = null;
        KinectHelper.CopySkeleton(skToBeScaled, ref newSkeleton);
        SkeletonPoint hipCenterPosition = newSkeleton.Joints[JointType.HipCenter].Position;
        foreach(BoneOrientation bo in skToBeScaled.BoneOrientations)
        {
            distanceToScale = Distance3DBetweenJoints(skToBeScaled.Joints[bo.StartJoint], skToBeScaled.Joints[bo.EndJoint]);
            distanceDestiny = Distance3DBetweenJoints(skDestiny.Joints[bo.StartJoint], skDestiny.Joints[bo.EndJoint]);

            if(distanceToScale > 0 && distanceDestiny > 0)
            {

                factor = (distanceDestiny / distanceToScale) * additionalFactor;


                newJoint = skToBeScaled.Joints[bo.EndJoint]; // escaling only the end joint as the BoneOrientatios starts from HipCenter, i am scaling from center to edges.

                factor = SmoothenFactor(newJoint.JointType, factor, weight);

                pos = new SkeletonPoint()
                {
                    X = (float)((newJoint.Position.X - hipCenterPosition.X) * factor + hipCenterPosition.X),
                    Y = (float)((newJoint.Position.Y - hipCenterPosition.Y) * factor + hipCenterPosition.Y),
                    Z = (float)((newJoint.Position.Z - hipCenterPosition.Z) * factor + hipCenterPosition.Z)
                };
                newJoint.Position = pos;
                newSkeleton.Joints[bo.EndJoint] = newJoint;
            }
        }
        return newSkeleton;
    }

I also modified your ScaleToMatch method as you see. There was a need to move joints in relation to HipCenter position. Also new positions are saved to a new Skeleton instance so they are not used in further vector calculations.

Experiment with the weight but since our bones length is constant you can use big numbers like 100 and more to be sure that wrong Kinect readings do not disturb the correct scale.

Here's an example of how it helped with scaling HandRight joint position:

enter image description here

The weight was set to 500. The resulting factor is supposed to be around 2 (because the base skeleton was purposely downscaled by a factor of 2).

I hope it helps!

Kinect skeleton Scaling strange behaviour

2 Answers2

Linked