-1

I'm trying to make a theremin using a Leap Motion controller through Processing (Java). So far, I've come very close. I am able to make the pitch and amplitude change by keeping one hand very still over the controller, while the other moves. I was hoping to make it so that either one hand is able to do that, or the left hand controls amplitude while the right controls pitch.

As it is, you have to keep one hand completely still while the other moves. If you only use one hand, it will only update the tone when the hand fully leaves and reenters the controller's view. I'm wondering if anyone can tell me why this is happening, and what I can do to make it update with the hand's position consistently.

Here's the code:

import processing.sound.*;
import de.voidplus.leapmotion.*;

LeapMotion leap;

ArrayList<PVector> points; 
PVector fp;

float freq;
float amp;

TriOsc tri;
SinOsc sin;
SqrOsc sqr;
SawOsc saw;

void setup () {

  size(640,800);

  leap = new LeapMotion(this);
  points = new ArrayList<PVector>();

  tri = new TriOsc(this);    
  sin = new SinOsc(this);
  sqr = new SqrOsc(this);
  saw = new SawOsc(this);

}

void leapOnInit() {
  // println("Leap Motion Init");
}
void leapOnConnect() {
  // println("Leap Motion Connect");
}
void leapOnFrame() {
  // println("Leap Motion Frame");
}
void leapOnDisconnect() {
  // println("Leap Motion Disconnect");
}
void leapOnExit() {
  // println("Leap Motion Exit");
}

void draw() {

  tri.freq(freq);
  tri.amp(amp);
  tri.play();

    for (Hand hand : leap.getHands()) {

       fp   = hand.getPosition(); 
       if (fp.z <= 30) {
       points = new ArrayList<PVector>();
    }

    else if (fp.z > 30) {
       points.add(new PVector(fp.x, fp.y));
    }
  }

  for (int i = points.size()-1; i >= 0; i--) {
     PVector p = points.get(i);
     amp = map(width-p.x, 0, width, 1, .01); //Volume based on x-axis
     freq = map(height-p.y, 0, height, 40, 880); //Pitch based on y-axis
  }
}
Nick Banducci
  • 15
  • 5
  • 1
    I'd also like to mention that I did some research into the issue at the following websites: (https://github.com/nok/leap-motion-processing) (https://developer-archive.leapmotion.com/getting-started/javascript/developer-guide#Leap) – Nick Banducci Dec 10 '18 at 05:38
  • 1
    Are you *sure* this is JavaScript code? It really looks like Java code.... – Claies Dec 10 '18 at 05:50
  • My mistake, yes you are right. I've edited it to reflect that it's Java, not JavaScript. – Nick Banducci Dec 10 '18 at 06:05
  • if that's the case, then the second link in your first comment might be a bit misleading as well, since it's the developer portal for JavaScript, which might give you completely inappropriate or broken code for the platform you are actually working in. – Claies Dec 10 '18 at 06:06
  • That is a fair point. Upon realizing that mistake, I looked up this link instead: (https://developer-archive.leapmotion.com/documentation/java/index.html) However, I haven't found anything yet that I think would solve this issue. – Nick Banducci Dec 10 '18 at 06:31
  • Your calculations aren't right. You are iterating through the `hands` and for each hand, you are recording it's position as a new vector, which means you will always have two vectors in your `points`. then you are iterating through points, and setting `amp` and `freq`, only to rewrite their values with the values of the second vector. – Claies Dec 10 '18 at 06:44
  • you should instead consider fleshing out your detection model a bit, using things like `hands.leftmost()` / `hands.rightmost()` to find the one on the left or right side of the view, or `hand.isLeft` / `hand.isRight` to determine if it's a left hand or right hand (no matter which side of the view), to allow you to, say, only calculate X from the left vector and Y from the right vector. – Claies Dec 10 '18 at 06:48
  • Wow. When you put it like that, it makes a lot of sense. I would love to give these upvotes for helpfulness, though it's not giving me the option. – Nick Banducci Dec 10 '18 at 06:53
  • well, I still think I'm on the right track, but even further review of the code shows you trying to reset `points` based on Z for each hand, which I think makes it even less clear from frame to frame what is actually going on. – Claies Dec 10 '18 at 06:53
  • yes, you need a minimum amount of points on the site to vote on comments; it's fine though... I'm not really prepared to post an answer, because I haven't actually written out any code to prove my thoughts here. – Claies Dec 10 '18 at 06:54
  • Based off what you commented earlier, I'm thinking that it will be best to create my own interaction box in the code, so that I can define the coordinates of the controller's view manually (https://developer-archive.leapmotion.com/documentation/v2/unity/api/Leap.InteractionBox.html#csharpclass_leap_1_1_interaction_box). I can then use what you mentioned before to split both hands into their own vectors, so that "amp" is determined by the left hand's X position and "freq" is determined by the right hand's Y-position. – Nick Banducci Dec 10 '18 at 20:42
  • Unfortunately, the more I look into this matter, the more I realize that Leap Motion's guide to setting up an interaction box isn't very helpful. Its example uses classes that are not defined elsewhere in the guide. There are also no good examples online of how to set one up in Java. The closest I got imported a lot of outside material, and I'm not sure it was even done on Processing (https://xiaotingrunning.wordpress.com/2015/03/23/leap-motion-tutorial-java9-interaction-box/). If you have any ideas, it would be appreciated. – Nick Banducci Dec 11 '18 at 22:46

1 Answers1

0

Okay, I got it. Let it be known, by the way, that the documentation on Leap Motion's SDK uses a COMPLETELY DIFFERENT LIBRARY from the one that is actually imported when you select "Sketch>Import Library...> Leap Motion for Processing" on Processing. I spent half a day trying to figure out why their coding examples used classes without telling you what were in them...

The documentation was frustrating to use, so I didn't use it or make my own interaction box. Instead, I toyed around with the code I had and found out that there was an "else" statement messing up my readings. Once I got rid of that, the darn thing worked like a charm. I even got it to separate "amp" and "freq" to my left and right hands!

Here's what it looks like:

import processing.sound.*;
import de.voidplus.leapmotion.*;

LeapMotion leap;
Theremin leapTheremin;

ArrayList<PVector> points; 
PVector handPos;

TriOsc tri;

void setup () {

  size(640,800);

  leap = new LeapMotion(this);
  points = new ArrayList<PVector>();

  tri = new TriOsc(this);    

  leapTheremin = new Theremin(tri);

}

void draw() {

  leapTheremin.renderSound();

  for (Hand hand : leap.getHands()) {

      handPos = hand.getPosition(); 
      boolean handIsLeft = hand.isLeft();
      boolean handIsRight = hand.isRight();

      if (handPos.z <= 75) {
       points = new ArrayList<PVector>();
       points.add(new PVector(handPos.x, handPos.y));
       background((handPos.x)/2.5,(width-handPos.x)/3,handPos.y-(height/3.5));
      }

       if (hand.isRight()) {
        leapTheremin.setPitch();
   }

      if (hand.isLeft()) {
        leapTheremin.setVolume();
      }   
   }
}

class Theremin {

float freq;
float amp;
int sound;

Theremin (TriOsc tri_g) {

  setPitch();

  sound = 1;

  tri = tri_g;

}

  void setPitch () {

    for (int i = points.size()-1; i >= 0; i--) {
    PVector p = points.get(i);
    freq = map((height-handPos.y)+10, 0, height, 40, 880); //"upright antenna", aka "the right one that controls pitch"
    // To match the colors with the moaods of the pitches   

    }   
  }

  void setVolume() {

    for (int i = points.size()-1; i >= 0; i--) {
    PVector p = points.get(i);
    amp = map(width-p.x, 0, width, 1, .01); //"loop antenna", aka "the left one that controls volume" 

    }
  }

  void renderSound() {
    tri.freq(freq);
    tri.amp(amp);
    tri.play();

  }
}
Nick Banducci
  • 15
  • 5