1

I have two videos which I want to fuse. I plan to take the visible light video, capture frames, and convert the frames to YUV. Before fusing this with the IR video, I want to extract the Y component, put it in another image(variable) of type Mat of the same size and fuse that with the luminance of IR frame. After fusion, I want to put the Cb and Cr back to get the colors.

I saw this link: Getting the Y and Cb , Cr values of a frame in openCV c++

But I couldn't figure out how I can create my own matrices with this method of extracting Y, Cb and Cr.

Any ideas how I can do this?

Community
  • 1
  • 1
st1046
  • 23
  • 1
  • 6

1 Answers1

3

look at split() and merge

Mat vis_img; // from cam

Mat ycrcb;
cvtColor( vis_img, ycrcb, CV_BGR2YCRCB ); // am i right assuming rgb input even ?

Mat channels[3];
split( ycrcb, channels );

// y is channels[0], cb is channels[1], cr, well...

// some operations on channels later..
// once you 'fused' your luminance channels into one, you can re-assemble the image
// from the channels:

Mat new_channels[3] = { lum_fused, channels[1], channels[2] };

Mat new_ycrcb;
merge( new_channels, new_ycrcb, 3 );
berak
  • 39,159
  • 9
  • 91
  • 89
  • Thanks, but I'm new to this so just asking a few points: When you created a new Mat named channels[3], that is a 1D matrix right? Now it won't be of the same size as ycrcb and I wouldn't be able to fuse it with another frame ? Does that make sense ? Or am I supposed to somehow get this Y for every pixel and then create another matrix? – st1046 Jun 25 '14 at 09:33
  • no, `Mat channels[3]` is an array of 3 empty Mats. they will get filled in the split() function. each of them will hold a 2d Mat the same size of the input img, with 1 color channel later – berak Jun 25 '14 at 09:35
  • 1
    You can also use 'std::vector channels'. You can then acces the sperate channels by using channels.at(number). – Nallath Jun 25 '14 at 09:47
  • @Nallath, is that a substitution for the split() function? I'd still need the initialisation : Mat channels[3] or am I completely wrong here? – st1046 Jun 25 '14 at 12:42
  • You can use the vector instead of the cv::Mat channels[3] as input for the CV::split. – Nallath Jun 25 '14 at 13:13
  • @berak, That works perfectly. I now have an image of type Mat which is a fusion of - Luminance of image_vis and Grayscale version of image_ir. I want to take the luminance of the NEW/FUSED image now. Is that possible with the same method? – st1046 Jun 25 '14 at 15:31
  • @st1046, don#t know if i understand you right, you got 1 luminosity (from cam viaycrcb), another from grayscale(yes, that's luminosity/intensity, too). and now ? the main difference is, that they come from different cameras, right ? really, no idea, what you're up to. – berak Jun 25 '14 at 18:11
  • @berak Apologies I didn't explain it quite right. Yes, I have 2 different videos, one from a visible light camera and the other from infra red. I am trying to capture frames from both videos, and fusing them together to get a better video. I plan to convert both frames from RGB to YUV (assuming both are RGB when I capture them), then use your method to extract the luminance, fuse the luminance of both , and then merge the final (fused) image with the Cr and Cb of the visible light image to get colors! – st1046 Jun 26 '14 at 08:54
  • ah, ok. so, 'fuse the luminance is the problem'. what would that be ? the average of both ? would be as simple as : `Mat lum = (lum_ir + lum_y) / 2;` – berak Jun 26 '14 at 08:58
  • @berak So fusing the luminance is fine, I've done that, I'm having problems with merging the Cr and Cb of the visible light with the fused image/mat ! – st1046 Jun 26 '14 at 09:03
  • I was trying somethinglike this but my way of putting values into new_channels was wrong, this is awesome cheers! – st1046 Jun 26 '14 at 09:16