not able to move mouse and capture gesture using Leapmotion controller in scala

Question

I'm trying to use leapmotion jar to replicate mouse motion with gesture and hand movement, i've created two methods

def executeGesture(gesture: Gesture) = {

    val robot = new Robot();
    gesture.match {
    case Gesture.Type.TYPE_CIRCLE => {
        println("CIRCLE IT IS")
        val circle = new CircleGesture(gesture);

        if (circle.pointable().direction().angleTo(circle.normal()) <= Math.PI / 4) { // Clockwise if angle is less than 90 degrees
            //              robot.mousePress(InputEvent.BUTTON1_MASK)
            //              robot.mouseRelease(InputEvent.BUTTON1_MASK)
            //              robot.mousePress(InputEvent.BUTTON1_MASK)
            //              robot.mouseRelease(InputEvent.BUTTON1_MASK)

        } else {

        }
    }
    case Gesture.Type.TYPE_SWIPE => {
        val swipe = new SwipeGesture(gesture)

        if (swipe.direction().getX() > 0) {
            println("SWIPE Right")

        } else {
            println("SWIPE Left ")
            robot.keyPress(KeyEvent.VK_ALT);
            robot.keyPress(KeyEvent.VK_F4);
            robot.keyRelease(KeyEvent.VK_ALT);
            robot.keyRelease(KeyEvent.VK_F4);
        }
    }
    case Gesture.Type.TYPE_SCREEN_TAP => {
        val ScreenTap = new ScreenTapGesture(gesture)
        println("Screen Tap " + ScreenTap.id())
    }
    case Gesture.Type.TYPE_KEY_TAP => {
        val KeyTap = new KeyTapGesture(gesture)
        println("Key Tap " + KeyTap.id())
        robot.mousePress(InputEvent.BUTTON1_MASK)
        robot.mouseRelease(InputEvent.BUTTON1_MASK)
    }
    case _ => println("Something ELSE!!. .")
    }
}

def executeMovement(frame: Frame) {
    val robot = new Robot
            val finger = frame.fingers().get(0)
            val gd = GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice()
            val Xwidth = gd.getDisplayMode().getWidth()
            val Xheight = gd.getDisplayMode().getHeight()

            val tipVelocity = finger.tipVelocity().magnitude().toInt
            val position = finger.tipPosition()

            if (tipVelocity > 2) {
                prevPoint.x = nextPoint.x
                        prevPoint.y = nextPoint.y

                        val mouseX = (Xwidth + Math.round(position.getX() * (Xwidth / 100)))
                        val mouseY = ((Xheight - (0.0F + position.getY() * 4.0F - Xheight / 5))).toInt

                        nextPoint.x = mouseX
                        nextPoint.y = mouseY

                        val diffx = (prevPoint.x - nextPoint.x).abs
                        val diffy = (prevPoint.y - nextPoint.y).abs

                        if ((diffx > 4) & (diffy > 4)) {
                            robot.mouseMove(nextPoint.x, nextPoint.y)
                            //        moveMouse(prevPoint.x, prevPoint.y, nextPoint.x, nextPoint.y, 200, 30)
                        }
            }
}

Executemovement moves mouse in the direction of your hand, when i comment out executemovement, the executeGesture recognizes all the gestures, but when i run both these methods, it does not detect the Key_tap and Screen_tap events. . And i'm not able to understand the reason behind it

score 0 · Answer 1 · answered Sep 10 '13 at 17:07

I suspect that you are skipping frames when you run both functions. If your onFrame handler doesn't return quickly enough, the Leap Motion software will skip frames until it is done. The Screen and KeyTap gestures only appear in a single frame, so it is easy to miss them if your code is dropping frames. The way around this is to save a reference to the last frame that your code processed and pass that into the Frame.gestures(sinceFrame) function. This gives you all the gestures objects that have been generated between the sinceFrame and the current frame.

not able to move mouse and capture gesture using Leapmotion controller in scala

1 Answers1