Transform irregular quadrilateral to rectangle in python matplotlib

Question

I have this data from a video.

Vertices of a rectangle
Points of an animal tracking inside the rectangle.

Due to image deformation, my "rectangle" is not regular. I want to transform the data in order to plot them in matplotlib as a rectangle.

Is there an easy method?

This is the maze and trancking. I decompose it into five quadrilaterals

This will require more than just matplotlib. Do you have the data for the vertices at each frame? — wgwz, Oct 22 '15 at 14:12
The vertices are fixed. The camera and the maze are in the same position throughout the experiment — heracho, Oct 22 '15 at 17:19
Could you share an image? Either directly here or a link to one. — wgwz, Oct 22 '15 at 19:11
What do you mean by "regular quadrilateral?" A regular quadrilateral is a square, but I don't suppose you mean that. Do you mean a rectangle with sides parallel to the axes? Do you just want to cut the figure up into five rectangles along the dotted lines in the small central square, and then rotate the resulting rectangles? What is the format of your data? The problem itself sounds easy, if my guesses about it are correct, but I need some more specifics, please. — saulspatz, Oct 23 '15 at 12:56
Your right, I need 4 rectangles and the central square. My data are numpy arrays. — heracho, Oct 23 '15 at 13:11
In the picture, there are dots connected by line segments. I take it that the dots are the data in your arrays. Is that correct? Also, what is the shape of your numpy arrays? — saulspatz, Oct 23 '15 at 13:40
exactly. The shape of the vertices are (12,2) and for the dots are (2, #dots). I have to deal with the problem of transform the coordinates for each rectangle without loose the position in the trajectory, but for now I'm stuck on the transformation problem. — heracho, Oct 23 '15 at 14:41

Mathias Rav · Accepted Answer · 2015-11-23T11:03:06.033

You can use skimage.transform.ProjectiveTransform from scikit-image to transform coordinates inside your quadrilateral into the local square space [0, 1] × [0, 1].

For more info on how to apply linear algebra to solve this problem, see ProjectiveTransform.estimate or "Projective Mappings for Image Warping" by Paul Heckbert, 1999.

Suppose you have the corners of your quadrilateral in clockwise order:

bottom_left = [58.6539, 31.512]
top_left = [27.8129, 127.462]
top_right = [158.03, 248.769]
bottom_right = [216.971, 84.2843]

We instantiate a ProjectiveTransform and ask it to find the projective transformation mapping points inside the quadrilateral to the unit square:

from skimage.transform import ProjectiveTransform
t = ProjectiveTransform()
src = np.asarray(
    [bottom_left, top_left, top_right, bottom_right])
dst = np.asarray([[0, 0], [0, 1], [1, 1], [1, 0]])
if not t.estimate(src, dst): raise Exception("estimate failed")

Now, the transformation t is ready to transform your points into the unit square. Of course, by changing dst above, you can scale to a different rectangle than the unit square (or even to an entirely different quadrilateral).

data = np.asarray([
    [69.1216, 51.7061], [72.7985, 73.2601], [75.9628, 91.8095],
    [79.7145, 113.802], [83.239, 134.463], [86.6833, 154.654],
    [88.1241, 163.1], [97.4201, 139.948], [107.048, 115.969],
    [115.441, 95.0656], [124.448, 72.6333], [129.132, 98.6293],
    [133.294, 121.731], [139.306, 155.095], [143.784, 179.948],
    [147.458, 200.341], [149.872, 213.737], [151.862, 224.782],
])
data_local = t(data)

We plot the input data and the transformed data to see the transformation working:

import matplotlib.pyplot as plt
plt.figure()
plt.plot(src[[0,1,2,3,0], 0], src[[0,1,2,3,0], 1], '-')
plt.plot(data.T[0], data.T[1], 'o')
plt.figure()
plt.plot(dst.T[0], dst.T[1], '-')
plt.plot(data_local.T[0], data_local.T[1], 'o')
plt.show()

It's work!!!. I only had problems with the line _if not t.estimate(src, dst): raise Exception("estimate failed")_ It gave the Exception. I only put _t.estimate(scr, dst)_ and it run. Thanks — heracho, Oct 23 '15 at 15:48

score -1 · Answer 2 · answered Oct 22 '15 at 19:10

Here is a tool you can use for corner detection. Note that in the example they deal with warping and affine transformations as well. Here is a basic example of corner detection. I am assuming that you do not have coordinates for the rectangles in your images.

import matplotlib.pyplot as plt
from skimage.feature import corner_harris, corner_subpix, corner_peaks

image = plt.imread('test.jpg')

coords = corner_peaks(corner_harris(image), min_distance=2)
coords_subpix = corner_subpix(image, coords, window_size=13)

fig, ax = plt.subplots()
ax.imshow(image,cmap=plt.cm.gray)
ax.plot(coords[:,1],coords[:,0],'.b',markersize=5)
plt.title("Example of corner detection")
ax.axis((0,800,800,0))
plt.xlabel('x (pixels)')
plt.ylabel('y (pixels)')

This outputs this image, you can see the quadrilateral was my image and skimage has found the corners (blue dots):

I will leave the transformation into a rectangle up to you. The AffineTransformation tool will do the trick. It will shift the points to form a rectangle. If I make any progress on this I will add to the post.

Maybe I misspoke, I have the points. Those are extracted manually in a GUI that uses Opencv. — heracho, Oct 22 '15 at 19:40
No, I made an assumption and was wrong. [This will still help you.](http://scikit-image.org/docs/dev/api/skimage.transform.html?highlight=hough_line#affinetransform). Or you could just look into rotation matrices through `numpy.linalg`. I think that is where the rotation matrices are.. — wgwz, Oct 22 '15 at 20:25

Transform irregular quadrilateral to rectangle in python matplotlib

2 Answers2

Linked