0

I've been trying to figure out how to get a birds-eye view of a scene by using a homography and then warping the image. The image that I am trying to warp is linked below, with the points selected with blue circles around them. I have seen advice from similar posts that I need to make sure the points are ordered correctly, and I have tried different orderings of the object points, but the result still has the same weird error.

The image points were selected manually, and you can see that the floor tile is close to a square in the output, as desired.

I believe the problem is with the homography matrix, as when the homography matrix is applied to the corners of the image, they end up at the pixel locations that they are in the output. As all corners are in the image, the standard method of calculating the corners of the image to get the full result as described in this post are not helpful.

Any help would be appreciated! The input image is: enter image description here

And the output image is: enter image description here

The code used to produce this is included below.

import cv2
import numpy as np

img = cv2.imread('test4_frame.png')
img_circled = img.copy()
im_pts = np.array([[659, 618], [779, 662], [542, 654], [661, 707]], np.float32)

for pt in im_pts:
  cv2.circle(img_circled, (pt[0], pt[1]), 10, 255)

cv2_imshow(img_circled)
obj_pts = np.array([[1500, 1500], [1550, 1500], [1500, 1550], [1550, 1550]], np.float32)
H = cv2.findHomography(im_pts, obj_pts)[0]
out = cv2.warpPerspective(img, H, (4000, 4000))
cv2_imshow(out)
Nick
  • 4,820
  • 18
  • 31
  • 47
Nate Armstrong
  • 101
  • 1
  • Seems like a pacman effect, move the destination point to the left bottom corner: `obj_pts = np.array([[3500, 3500], [3550, 3500], [3500, 3550], [3550, 3550]], np.float32)` – iGian Jul 17 '20 at 19:09
  • Wow, I never heard of the name Pacman-Effect but I'm always going to use it when talking about coordinate system confisuion from now on :) – vatbub Aug 04 '23 at 09:58

1 Answers1

0

After another couple hours banging my head against this problem, I noticed that the issue is that the plane I am interested in will never reach the points in the image above the vanishing point. Thus, the points which are above the vanishing point have undefined behavior, as it cannot be projected onto the plane, which is what the issue was caused by.

I solved the issue by roughly estimating the vanishing point and cropping the image a bit below there, while adjusting the image points accordingly, the results of which are here.

van_point_est = 250
img = img[van_point_est:, :]
for i in range(len(im_pts)):
  im_pts[i, 1] -= van_point_est
Nate Armstrong
  • 101
  • 1