OpenCV, dynamic modifying of webcam display

Question

Sorry if the title gave you the wrong idea, I tried to make it as brief as possible. In short, what I am trying to do is detect a face with Viola-Jones algorithm (already implemented), save it in a separate image, convert that image to grayscale, then slap the grayscale image back into its' original position, resulting in a webcam display with all faces (and any false positives, I suppose) colored gray and surrounded by a green rectangle. However, I get the following error message:

Unhandled exception at 0x771115de in proba.exe: Microsoft C++ exception: cv::Exception at >memory location 0x003ef2c8..

This is my code (the relevant part), any suggestion/advice would be appreciated:

face_cascade.detectMultiScale( frame_gray, faces, 1.1, 2, 0|CV_HAAR_SCALE_IMAGE, Size(30, 30) );
for( int i = 0; i < faces.size(); i++ )
{
Point pt1(faces[i].x + faces[i].width, faces[i].y + faces[i].height);
Point pt2(faces[i].x, faces[i].y); 

Rect myROI(pt1, pt2);
Mat croppedImage;
Mat(frame, myROI).copyTo(croppedImage);
cvtColor(croppedImage, croppedImage, CV_BGR2GRAY ); //the last four lines process the image

croppedImage.copyTo(frame(Rect(pt1, croppedImage.size()))); //this should copy the image back into its' original location

rectangle(frame, pt1, pt2, cvScalar(0, 255, 0, 0), 1, 8, 0);  
}
//-- Show what you got
imshow( window_name, frame );

And sorry if I'm missing the obvious answer.

Answer 1

Your cropped grayscale image croppedImage is a 1 channel image but you are trying to overlay it onto 3 channel RGB image frame. In other words, the function copyTo in

croppedImage.copyTo(frame(Rect(pt1, croppedImage.size())));

expects croppedImage to have the same number of channels as frame. This is why you are getting the error.

EDIT To solve your issue you may try convert your grayscale cropped image back to RGB format (it will still look like the grayscale image). Something like

cvtColor(croppedImage, croppedImage, CV_BGR2GRAY );  // to grayscale
cvtColor(croppedImage, croppedImage, CV_GRAY2BGR );  // to RGB
croppedImage.copyTo(frame(Rect(pt1, croppedImage.size())));