3

I am using a leap motion sensor with my node app to detect gestures. With my current setup, the gesture detection is working but not as accurately as I need it to be. When I perform a swipe gesture the leap motion does not always send a swipe gesture object and when it does detect a swipe it often will send multiple gestures. For example, when swiping left, it will return a left swipe as well as an up swipe. I am wondering if this is an issue with my code or my leap motion sensor.

Here is my leap controls code:

var controller = Leap.loop({ inNode: true, enableGestures: true }, function(frame) {

  // finger location tracking
  if(frame.hands.length > 0) {
    var iBox = frame.interactionBox;
    var pointable = frame.pointables[0];

    if (pointable) {
      var leapPoint = pointable.stabilizedTipPosition;
      var normalizedPoint = iBox.normalizePoint(leapPoint, true);
      let mouseControlY = (1 - normalizedPoint[1]) * appHeight; // with leap facing up

      // invert directions for mirror
      let invertMouseControlX = (1 - normalizedPoint[0]) * appWidth;

      updateMouseCords(invertMouseControlX, mouseControlY)
    }
  }

  // finger gesture tracking
  if(frame.valid && frame.gestures.length > 0){
    frame.gestures.forEach(function(gesture){
        switch (gesture.type){
          case "screenTap":
              console.log("Click");
              break;
          case "swipe":
              handleSwipe(gesture);
              break;
        }
    });
  }
});

function handleSwipe (swipe) {
  let io = getIO();
  
  if(swipe.state === 'stop'){
      if (swipe.direction[0] > 0){
        // user swiped right
        swipeDirection = 'right'
      } else {
        // user swiped left
        swipeDirection = 'left'
      }
      
      if(swipe.direction[1] > 0) { 
        // user swiped up
        swipeDirection = 'up'
      }  

      // send swipe direction to frontend
      io.emit('swipeData', swipeDirection);
  }
}

I only need to detect a left, right, or up swipes and possibly the screen tap gesture. Is there anything I can do to optimize my code or is the issue likely my sensor?

whoMe
  • 227
  • 2
  • 13
  • which SDK are you using? the latest version doesnt support web apps anymore and gestures have been removed for even longer. I've had trouble myself also with this. Failed to find the right SDK and then the right leapjs version (1.0.0) to get gestures to work. I managed to get something basic with Orion (4.1) and the npm module `leapjs-gestures` but had to fix the code since interactionBox was removed, and now not all gestures are recognised. If you have a better JS / web app setup I'd love to hear – Flion Nov 30 '21 at 06:45

1 Answers1

0

I am not an expert myself, but I have prevented multiple detections by using the setTimeout() function, which is set to a few milliseconds where no gesture detection can occur.

The JavaScript documentation also mentions the attributes Gesture.duration and Gesture.type. Perhaps you can achieve better results by using them.

It is also worth noting, that Gestures are unfortunately deprecated in version 3.0. However, I am not sure if this is due to the problems you described or if they will be fundamentally revised in later versions. I would also be interested in an update on this topic.

Isabella Molterer
  • 1