5

I am trying to segment the hand from this depth image:

enter image description here

I tried watershed, region growing, grabcut, but all of them failed mainly because there is not a clear edge. I tried to sharp the image as well, but it didn't give me good results either.

Shai
  • 111,146
  • 38
  • 238
  • 371
Jäger
  • 71
  • 10
  • 2
    is this your original resolution and color depth? if not, please provide original sample image so a method can be tested before it will be proposed. – Micka Aug 05 '15 at 13:03
  • @Micka I updated to the original color depth (and this is the original resolution of my DS325 camera). The only thing is that the input was converted from float (varying from -1 to 0.6) to uchar to be saved as jpg. If you think is better to send in float format, I can link a txt file. – Jäger Aug 05 '15 at 13:40
  • link the txt file in addition please =) – Micka Aug 05 '15 at 13:42
  • 1
    @Micka Its [here](https://dl.dropboxusercontent.com/u/5200032/output.txt) (the size is 320x240). And thanks for the help by the way! – Jäger Aug 05 '15 at 13:56
  • 1
    with some prior knowledge it might be possible (still hard) to segment most of the hand, but I have no idea at all how to get the thumb... – Micka Aug 05 '15 at 14:40
  • Yes, for now I was using color information, but I don't like it since any table with skin color give me bad results. And since I can see the borders with my eyes, there must be a way to solve this... – Jäger Aug 05 '15 at 14:59
  • 2
    probably your brain enhances the thumb outlines because your brain already detected the rest of the hand ;) It is quite challenging to use the humand brain as a reference for segmentation algorithms! – Micka Aug 05 '15 at 15:11

2 Answers2

4

This might not be the answer you were hoping for, but it might help you step forward a bit. Since I only provide algorithmic hints I will use Matlab rather than opencv.

Since this is not an ordinary intensity image, but rather depth image, you should use the implied geometry of the scene. The key assumption that can help you here is that the hand is resting on a surface. If you can estimate the surface equation, you can detect the hand much easier.

[y x] = ndgrid( linspace(-1,1,size(img,1)), linspace(-1,1,size(img,2)) );
X = [reshape(x(101:140,141:180),[],1), reshape(y(101:140,141:180),[],1), ones(1600,1)];
srf=(X\reshape(img(101:140,141:180),[],1)); %// solving least-squares for the 40x40 central patch

 aimg = img - x*srf(1) - y*srf(2) - srf(3); %// subtracting the recovered surface

Using median filter to "clean" it a bit, and applying a simple thereshold

 medfilt2(aimg,[3 3]) < -1.5

Yields

enter image description here

Not exactly what you were hoping for, but I think it's a step forward ;)

PS,
You might find the work of Alpert, Galun, Nadler and Basri Detecting faint curved edges in noisy images (ECCV2010) relevant to your problem.

Shai
  • 111,146
  • 38
  • 238
  • 371
  • thank you for the answer! Actually I had this same idea. With RANSAC I extracted the plane very well, but I still miss the fingertips (which is what I'm looking for). And thanks for pointing out the paper, I'll take a look into it. – Jäger Aug 06 '15 at 12:05
1

Please check my code implementation.

void applyClahe(const cv::Mat& src, cv::Mat& dst)
{
    cv::Mat lab;
    cv::cvtColor(src, lab, cv::COLOR_BGR2Lab);

    vector<cv::Mat> labChannels;
    cv::split(lab, labChannels);

    auto clahe = cv::createCLAHE();
    cv::Mat cl; clahe->apply(labChannels[0], cl);
    cl.copyTo(labChannels[0]);

    cv::merge(labChannels, dst);
    cv::cvtColor(dst, dst, cv::COLOR_Lab2BGR);
}

This function will give you the following result and good starting point. CLAHE Result

Dong He Zheng
  • 89
  • 3