Face detection with opencv, open kinect and kinect v2 in Processing

Question

i am trying build a face detection with the opencv and openkinect libraries. for the image input i want to use the xbox kinect v2. i am basing my code on the face detection example within the opencv library. i am working on a mac.

this is my code so far:

import gab.opencv.*;
import java.awt.Rectangle;

/* KINECT */
import org.openkinect.freenect.*;
import org.openkinect.freenect2.*;
import org.openkinect.processing.*;

OpenCV opencv;
Kinect2 kinect2;

Rectangle[] faces;

void setup() {
  opencv = new OpenCV(this, 640/2, 480/2);
  size(640, 480);
  // Kinectv2
  kinect2 = new Kinect2(this);
  kinect2.initVideo();
  kinect2.initDevice();

  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  
  faces = opencv.detect();
}

void draw() {
  opencv.loadImage(kinect2.getVideoImage());
  image(kinect2.getVideoImage(), 0, 0, 640, 480);

  noFill();
  stroke(0, 255, 0);
  strokeWeight(3);
  for (int i = 0; i < faces.length; i++) {
    rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
  }
}

the problem seems to be in the line "opencv.loadImage(kinect2.getVideoImage());" since the detection does not work. when working with the isight camera (using the build-in function "capture" and "video"-add-on) instead of kinect everything works perfectly fine.

can anyone help?

You're going to have to be much more specific. What exactly do you mean when you say the problem seems to be on that line? What exactly do you mean when you say it doesn't work? What do you expect to happen? What happens instead? Which line of code does the different behavior start on? — Kevin Workman, Sep 12 '16 at 14:41
opencv.loadImage(kinect2.getVideoImage()); //here i expect opencv to process the visual data recorded by the kinect which is NOT working. when i say "the problem has to be in that line" is that it could have something to do with opencv not being able to process the data it gets from the function "kinect2.getVideoImage()" and maybe need a different form of input. image(kinect2.getVideoImage(), 0, 0, 640, 480); // here i expect the output of the visual data recorded by the xbox kinect which IS working. — mindcraft, Sep 12 '16 at 22:49

score 1 · Answer 1 · edited May 23 '17 at 11:48

In the future, please try to provide an MCVE. That means starting over with a blank sketch and only adding enough code so we can see the problem. In your case, we don't need to see any kinect code. Just load a hardcoded image and pass that to OpenCV.

Here is an example sketch that does exactly that. I got the image url from the human Wikipedia page.

import gab.opencv.*;
import java.awt.Rectangle;

PImage image;
OpenCV opencv;

void setup() {
  size(500, 500);
  image = loadImage("https://upload.wikimedia.org/wikipedia/commons/thumb/2/27/A_young_Man_and_Lady.png/800px-A_young_Man_and_Lady.png");
  image.resize(width, height);

  opencv = new OpenCV(this, width, height);
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);
}

void draw() { 

  image(image, 0, 0);

  opencv.loadImage(image);
  Rectangle[] faces = opencv.detect();

  noFill();
  stroke(255, 0, 0);
  strokeWeight(3);
  for (Rectangle face : faces) {
    rect(face.x, face.y, face.width, face.height);
  }
}

OpenCV face detection

Notice that I'm calling the opencv.detect() function every frame. You're only calling it from setup(), which means you're only detecting faces in the very first frame.

If you still can't get it working, then you're going to have to do some debugging. Try to isolate your problem as much as possible. Get rid of the kinect code and just use a hardcoded image. Take a screen capture of the image being obtained through your kinect, and use that instead of the live stream. Work in smaller steps, that way you can post a more specific question when you get stuck (it's hard to help with general "this isn't working" questions- it's much easier to help with specific "I tried X, expected Y, but got Z instead" type questions). Good luck.

Thank you for your reply and your advices. Since I was not able to find a solution for this problem in time I switched over to developing under windows with the processing library "Kinect 2 for Windows" which works with the official Xbox Kinect SDK which incorporates a face-recognition as well and thus I dismissed openCV for face recognition. — mindcraft, Oct 13 '16 at 12:32
Actually trying things out with a hardcoded single image was my very first approach before asking here and of course it worked. My problem was that the data continuously received by the Kinect could not have been processed by openCV, at least not in the way I expected how it should have been implemented in the code. But what I haven´t tried so far is calling the opencv.detect() outside of the setup()-function. I´ll try that as soon as I finde the time to do so. Still, I appreciate your feedback! — mindcraft, Oct 13 '16 at 12:41

Face detection with opencv, open kinect and kinect v2 in Processing

1 Answers1