0

I have two images, one image which contains a box and one without. There is a small vertical disparity between the two pictures since the camera was not at the same spot and was translated a bit. I want to cut out the box and replace the hole with the information from the other picture.

I want to achieve something like this (a slide from a computer vision course)

enter image description here

I thought about using the cv2.createBackgroundSubtractorMOG2() method, but it does not seem to work with only 2 pictures.

Simply subtracting the picture from another does not work either because of the disparity. The course suggests using RANSAC to compute the most likely relationship between two pictures and subtract the area thaht changed a lot. But how do I actually fill in the holes?

Many thanks in advance!!

eyllanesc
  • 235,170
  • 19
  • 170
  • 241
rfn123
  • 151
  • 3
  • 12

2 Answers2

1

If you plant ot use only a pair of images (or only a few images), image stitching methods are better than background subtraction.

The steps are:

  1. Calculate homography between the two images.
  2. Warp the second image to overlap the second.
  3. Replace the region with the human with pixels from the warped image.

This link shows a basic example of image stitching. You will need extra work if both images have humans in different places, but otherwise it should not be hard to tweak this code.

Totoro
  • 3,398
  • 1
  • 24
  • 39
  • I followed your suggestion, and I encounter a problem warping the image. I manage to get the homography but only part of the image is warped correctly. I believe it's due to the parallax in direction of the scene (Z direction) of the two cameras. Is there a way to correct this? – rfn123 May 24 '17 at 15:51
  • One way out is to match feature points from the neighborhood of the region that you need to fill, so that the estimated homography is more accurate for that region. I think a good textbook on 3D Vision might contain a more sound approach. – Totoro May 25 '17 at 06:34
0

You can try this library for background subtraction issues. https://github.com/andrewssobral/bgslibrary

there is python wrappers of this tool.

Eren V.
  • 210
  • 1
  • 4
  • 12
  • thanks for the quick reply, does this library also work with two pictures instead of a video? – rfn123 May 22 '17 at 14:34