0

I am trying to find out the difference in 2 images.

Scenario: Suppose that i have 2 images, one of a background and the other of a person in front of the background, I want to subtract the two images in such a way that I get the position of the person, that is the program can detect where the person was standing and give the subtracted image as the output.

The code that I have managed to come up with is taking two images from the camera and re-sizing them and is converting both the images to gray scale. I wanted to know what to do after this. I checked the subtract function provided by OpenCV but it takes arrays as inputs so I don't know how to progress.

The code that I have written is:

cap>>frame; //gets the first image
cv::cvtColor(frame,frame,CV_RGB2GRAY); //converts it to gray scale
cv::resize(frame,frame,Size(30,30));  //re-sizes it

cap>>frame2;//gets the second image
cv::cvtColor(frame2,frame2,CV_RGB2GRAY); //converts it to gray scale
cv::resize(frame2,frame2,Size(30,30)); //re-sizes it

Now do I simply use the subtract function like:

cv::subtract(frame_gray,frame,frame);

or do I apply some filters first and then use the subtract function?

praxmon
  • 5,009
  • 22
  • 74
  • 121
  • _"it takes arrays as inputs so I don't know how to progress"_ the contents of an image are nothing but an array of bytes, right? Anyway, have you actually tried doing anything with `cv::subtract` yet? – Rook Jul 10 '12 at 10:27
  • No, when I saw the documentation I just added the subtract statement as a comment in the code. I don't need any filters to work on the images before I subtract them? And do I need to load the images into the memory before subtracting them? I read so somewhere, was not sure if I was necessary. – praxmon Jul 10 '12 at 10:32
  • 1
    You should perhaps search for more information on opencv image or background subtraction. A very quick search brings up [a related stackoverflow issue](http://stackoverflow.com/questions/9742591/opencv-issue-of-image-subtraction) for example. See how it is done elsewhere, and ask if you get stuck rather that resorting to asking for help before trying anything ;-) – Rook Jul 10 '12 at 10:39

2 Answers2

1

As others have noticed, it's a tricky problem: easy to come up with a hack that will work sometimes, hard to come up with a solution that will work most of the time with minimal human intervention. Also, much easier to do if you can control tightly the material and illumination of the background. The professional applications are variously known as "chromakeying" (esp. in the TV industry), "bluescreening", "matting" or "traveling matte" (in cinematography), "background removal" in computer vision.

The groundbreaking work for matting quasi-uniform backdrops was done by Petro Vlahos many years ago. The patents on its basic algorithms have already expired, so you can go to town with them (and find open source implementations of various quality). Needless to say, IANAL, so do your homework on the patent subject.

Matting out more complex backgrounds is still an active research area, especially for the case when no 3D information is available. You may want to look into a few research papers that have come out of MS Research in the semi-recent past (A. Criminisi did some work in that area).

Francesco Callari
  • 11,300
  • 2
  • 25
  • 40
0

Using the subtract would not be appropriate because, it might result in some values becoming negative and will work only if you are trying to see if there is a difference or not( a boolean true/false).

If you need to get the pixels where it is differing, you should do a pixel by pixel comparison - something like:

int rows = frame.rows;
int cols = frame.cols;
cv::Mat diffImage = cv::Mat::zeros(rows, cols, CV_8UC1);
for(int i = 0; i < rows; ++i)
{
    for(int j = 0; j < cols; ++j)
    {

        if(frame.at<uchar>(i,j) != frame2.at<uchar>(i,j))
            diffImage.at<uchar>(i, j) = 255;
    }
}

now, you can either show or save diffImage. All pixels that differ will be white while the similar ones will be in black

go4sri
  • 1,490
  • 2
  • 15
  • 29
  • frame.at() is a valid statement? I am using Visual Studio 2008 and opencv 2.3 and its throwing an error. – praxmon Jul 10 '12 at 10:38
  • Sorry! I missed the data type. I have edited the program to add `` the data type. – go4sri Jul 10 '12 at 10:40
  • There are alternatives to subtract provided by OpenCV, such as `cvAbsDiff`. Incidentally, if the images being compared are camera frames from a live scene rather than synthetically generated, the chance of pixel values being identical between two frames of the same subject are rather slim! – Rook Jul 10 '12 at 10:42
  • Aaannnd its working now. Just have to check the output though. Thanks. :) – praxmon Jul 10 '12 at 10:43
  • @Rook is right. If you are trying to learn OpenCV, this is a good start, but background removal is not a simple problem. There are a lot of sophisticated methods such as learning the background, averaging background etc. you might want to look those up. – go4sri Jul 10 '12 at 10:46
  • Do I really need to re-size the images? I mean they are from the same camera feed? And yeah I realize that but I have a project to complete so time constraints matter thus have to do stuff quickly. – praxmon Jul 10 '12 at 10:47
  • No - you don't. But, resizing has both advantages and disadvantages the advantage in that, by resizing you are sampling, and might get rid of some noise. However, you are also losing information. – go4sri Jul 10 '12 at 10:49
  • Thanks....ok the program is not running fully, some run time error. Will have to sort that out! Thanks again for help. – praxmon Jul 10 '12 at 10:52