3

We want to attach some UI and other items to the back of the articulated hand. Just trying to figure out how to do that. I have found how to turn on and off the hand visualizer through MixedRealityHandTrackingProfile but I'm trying to find the Unity Game Object I can parent the items to or at least a way to access the hand transform. Thanks for any pointers!

Jared Bienz - MSFT
  • 3,550
  • 17
  • 21

2 Answers2

3

Step 1: Select the object in the scene hierarchy that you want to follow your hand. Click “add component” in the inspector panel.

Step 2: Type in “RadialView” in the search box and you should see the RadialView solver appear. Click on it. You will see a few additional required scripts appear automatically. Note: it adds the solver handler script. Along with that, the Radial View script will show up as well just like the orbital script.

Step 3: Change the radial view to not follow the head but follow the left hand. Select the dropdown menu next to the “tracked object to reference” option. Then select “hand joint left” from the menu.

Step 4: As you may see, once you select the hand joint you can choose which part of the hand you want the cube to follow. There are a lot of options to use! For this example, we are going to use the wrist. So next to the option “tracked hand joint” click the dropdown menu and select wrist. Note: Not all joints, in this current version of the HoloLens2 can be tracked. This is a bug that may be fixed in the near future.

Now if you press play and try it out in your scene, you will see that the object does follow the wrist, but the object may lag a little bit behind and looks like it’s struggling to keep up. Now to fix this and make it so that the object is with the wrist at all time we must change a few things. Set the maximum and minimum distances to 0 so that the cube will not have any distance between it and the user’s wrist. Once set, the cube will be perfectly aligned with the wrist.

Sean Ong
  • 46
  • 2
  • Thank you Sean! This seems to work however I can't seem to get the box to rotate to align with the hand. I chose "Palm" and I chose a Reference Direction of "Object Orientated", but the cube seems to always be orientated toward ground. Is this just an issue in the simulator? I haven't actually deployed to the device yet. – Jared Bienz - MSFT Apr 09 '19 at 18:48
  • Yes, I believe it may be an editor issue. I experienced that objects don't respond to the simulated hand rotations, but they do the articulated hands on device. Try that and see if it helps. Also - when I tried the palm before, there was a bug, but the wrist seemed to track well, so you might explore that also. – Sean Ong Apr 09 '19 at 19:06
  • So, it turned out that there is an option on Radial View called " Orientate to Reference Direction". This is not checked by default and we needed to turn it on for the object to orientate correctly. – Jared Bienz - MSFT Apr 09 '19 at 20:19
0

In the latest mrtk_development branch as of PR 4532 you can also used the "Hand Constraint" component. You can see an example of how to use it at MixedRealityToolkit.Examples/Experimental/HandTracking/Scenes/HandBasedMenuExample.unity.

Have a look at Assets/MixedRealityToolkit.SDK/Experimental/Features/Utilities/Solvers/HandConstraint.cs for the implementation.

You can add this behavior by adding a "Hand Constraint" solver to the object that you wish to follow the hand.

The Hand Constraint component will also be available in the upcoming MRTK V2.0.0 RC2 release.

Julia Schwarz
  • 2,610
  • 1
  • 19
  • 25