cv2.perspectiveTransform() not accepting homography matrix

Question

Below are bookhomography-example-1.jpg and then bookhomography-example-2.jpg from a popular OpenCV blogpost about homography.

I can do the homography and warp the image, but the h or h[0] do not work when I try to use cv2.perspectiveTransform(pts, h) or cv2.perspectiveTransform(pts, h[0]). I've also tried converting the 2D array h[0] to a tuple of tuples, but no change. It's probably something simple, but I can't figure it out.

Error Message:

Traceback (most recent call last):

File "bookhomography stackexchange v00.py", line 36, in T_dst = cv2.perspectiveTransform(pts_dst, h) TypeError: m is not a numerical tuple

note: set False to True to induce failure. One of the two transform lines is the wrong direction but both fail.

import numpy as np
import matplotlib.pyplot as plt
import cv2

im_src = cv2.imread("bookhomography-example-2.jpg")
im_dst = cv2.imread("bookhomography-example-1.jpg")

im_srcrgb = cv2.cvtColor(im_src, cv2.COLOR_BGR2RGB)
im_dstrgb = cv2.cvtColor(im_dst, cv2.COLOR_BGR2RGB)

pts_src = np.float32([52, 376, 240, 528, 413, 291, 217, 266]).reshape(4, -1)
pts_dst = np.float32([56, 478, 387, 497, 376, 124, 148, 218]).reshape(4, -1)

h       = cv2.findHomography(pts_src, pts_dst)

print "type(h): ", type(h)
print "len(h): ", len(h)

print "type(h[0]): ", type(h[0])
print "len(h[0]): ", len(h[0])
print "h[0].shape: ", h[0].shape

shape   = im_src.shape[:2][::-1]

print h[0]

print "pts_src:"
print pts_src

print "pts_dst:"
print pts_dst

if False:
    T_dst = cv2.perspectiveTransform(pts_dst, h)
    T_src = cv2.perspectiveTransform(pts_src, h)

    print "T_src:"
    print T_src

    print "T_dst:"
    print T_dst

im_fin  = cv2.warpPerspective(im_src, h[0], shape)
im_finrgb  = cv2.cvtColor(im_fin, cv2.COLOR_BGR2RGB)

plt.figure()
plt.subplot(1, 3, 1)
plt.imshow(im_srcrgb)
x, y = pts_src.T
plt.plot(x, y, 'or', ms=8)
plt.subplot(1, 3, 2)
plt.imshow(im_dstrgb)
x, y = pts_dst.T
plt.plot(x, y, 'or', ms=8)
plt.subplot(1, 3, 3)
plt.imshow(im_finrgb)
x, y = pts_dst.T
plt.plot(x, y, 'or', ms=8)
plt.show()

Answer 1

See my answer here for the quick fix. TL:DR; The OpenCV function perspectiveTransform() takes points specified in an odd format, whereas findHomography() works with the format you have.

---

First note that findHomography() returns two values; the homography matrix, and a mask. From the docs:

cv2.findHomography(srcPoints, dstPoints[, method[, ransacReprojThreshold[, mask]]]) → retval, mask

The second return value is not the homography, hence why h[0] should be used. Alternatively you could write:

h, mask = cv2.findHomography(srcPoints, dstPoints)

or

h = cv2.findHomography(srcPoints, dstPoints)[0]

so that h only holds the homography for less confusion. And note that using h or h[0] as you've specified gives you different error messages:

Using h:

>>> T_dst = cv2.perspectiveTransform(pts_dst, h)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: m is not a numerical tuple

Using h[0]:

>>> T_dst = cv2.perspectiveTransform(pts_dst, h[0])
OpenCV Error: Assertion failed (scn + 1 == m.cols) in perspectiveTransform, file .../opencv/modules/core/src/matmul.cpp, line 2299
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cv2.error: .../opencv/modules/core/src/matmul.cpp:2299: error: (-215) scn + 1 == m.cols in function perspectiveTransform

The error messages are not really helpful to you here, unfortunately, because the actual problem is the way the points are specified. This is technically user error, but the docs (and the function itself) should probably be changed.

---

Anyways, the fix: add a channel to your pts vectors. See the difference:

>>> np.float32([52, 376, 240, 528, 413, 291, 217, 266]).reshape(4, -1)
array([[  52.,  376.],
       [ 240.,  528.],
       [ 413.,  291.],
       [ 217.,  266.]], dtype=float32)
>>> np.float32([52, 376, 240, 528, 413, 291, 217, 266]).reshape(4, 1, -1)
array([[[  52.,  376.]],

       [[ 240.,  528.]],

       [[ 413.,  291.]],

       [[ 217.,  266.]]], dtype=float32)

Luckily, findHomography() works with this format as well, so you don't have to use two different formats depending on which function you're using. To be safe, just always put points in this format for OpenCV functions.

>>> pts_src = np.float32([52, 376, 240, 528, 413, 291, 217, 266]).reshape(4, 1, -1)
>>> pts_dst = np.float32([56, 478, 387, 497, 376, 124, 148, 218]).reshape(4, 1, -1)
>>> h = cv2.findHomography(pts_src, pts_dst)[0]
>>> T_dst = cv2.perspectiveTransform(pts_dst, h)
>>> T_src = cv2.perspectiveTransform(pts_src, h)
>>> T_src
array([[[  56.,  478.]],

       [[ 387.,  497.]],

       [[ 376.,  124.]],

       [[ 148.,  218.]]], dtype=float32)
>>> T_dst
array([[[ 157.78089905,  588.9598999 ]],

       [[ 495.96539307,  365.68994141]],

       [[ 200.45231628,  -69.54611206]],

       [[  15.72697926,  204.0632019 ]]], dtype=float32)

The above produces no errors.