0

I'm doing a mosaic from a video in Opencv. I'm using this example for stitching the frames of the videos: http://docs.opencv.org/doc/tutorials/features2d/feature_detection/feature_detection.html. At the end I'm doing this for merging the new frame with the stitch created at the passed iteration:

Mat H = findHomography(obj, scene, CV_RANSAC);

static Mat rImg;
warpPerspective(vImg[0], rImg, H, Size(vImg[0].cols, vImg[0].rows), INTER_NEAREST);//(vImg[0], rImg, H, Size(vImg[0].cols * 2, vImg[0].rows * 2), CV_INTER_LINEAR);

static Mat final_img(Size(rImg.cols*2, rImg.rows*2), CV_8UC3);
static Mat roi1(final_img, Rect(0, 0, vImg[1].cols, vImg[1].rows));
Mat roi2(final_img, Rect(0, 0, rImg.cols, rImg.rows));

rImg.copyTo(roi2);
vImg[1].copyTo(roi1);

imwrite("stitch.jpg", final_img);
vImg[0] = final_img;

So here's my problem: obviously the stitch becomes larger at each iteration, so how can I resize it to make it fit in the final_img image?

EDIT Sorry but I had to remove images

bjorn
  • 338
  • 6
  • 18

1 Answers1

0

For the second question, what you observe is an error in the homography that was estimated. This may come either from:

  1. drift (if you chain homographies along the sequence), ie, small errors that accumulate and become large after dozens of frames
  2. or (more likely) because your reference image is too old with respect to your new image, and they exhibit too few matching points to give an accurate homography, but yet enough to find one that passes the quality test inside cv::findHomography().

For your first question, you need to add some code that keeps track of the current bounds of the stitched image in a fixed coordinate frame. I would suggest to choose the coordinates linked with the first image.

Then, when you stitch a new image, what you do really is to project this image onto this coordinate frame. You can compute first for example the projected coordinates of the 4 corners of the incoming frame, test if they fit into the current stitching result, copy it to a new (bigger) image if necessary, then proceed with stitching the new image.

sansuiso
  • 9,259
  • 1
  • 40
  • 58
  • Hi! at each stitch step, I update the reference image with the just stitched one, so I think I'm in the case that you called drift. If I'm in that case, how can I resolve? For the first question, I'll try to do what you say and let you know if it works! – bjorn Mar 25 '15 at 09:08
  • If you take the stitched image as reference, you may also lose in some cases some information (because of blur introduced by the interpolation, or scale differences between the stitched result and the current frame). An alternate possibility (probably with higher quality but more development overhead) would be to compute all the homographies in a first pass, then look for a stitched image optimized from all the input frames at the same time. – sansuiso Mar 25 '15 at 09:14
  • Mmm I understand but I can't do it because it's a stream and not a video file on pc.. For now I'm trying to resize the stitch, I will let you know if I succeed. . Thanks for the quick replies! – bjorn Mar 25 '15 at 10:54
  • Hi! I'm sorry but can you tell me how to track the bounds of the stitch or where I have to look? I'm very new to opencv so I don't know the framework very well! – bjorn Mar 31 '15 at 12:35
  • You have to do it (mostly) by yourself. The image-to-image transform comes as a homography H. This himography is applied to each pixel coordinates in the new frame to give its stitching coordinates, simply using (u,v)^T = H * (x,y)^T. Whenever you get a new frame, update the bounding box of the stitching result by checking if the (u,v) coordinates are inside or outside the previous bounding box. Initialize the bounding box as the first frame box (since it is not transformed). – sansuiso Mar 31 '15 at 13:38