Detecting perspective angle and performing perspective transform using OpenCV

Question

I have some content to be feature-detected using OpenCV. Before applying detection to the content itself, I need to detect the angle of perspective distortion and fix it. Probably I can use the idea of how QR-code does it - add several anchors to the image. There won't be any square-shaped content, so the source image may look like:

Source Image

The 3 squares will be our anchors.

When using OpenCV, as far as I understand, I should do these operations:

1. Apply grayscale scheme to the image using cvtColor function.
2. Apply blur using bluror GaussianBlur.
3. Apply threshold and Canny.
4. Use findContours and analyze the results, find the 3 squares by checking vertices count or similar.
5. Apply the perspectiveTransform and warpPerspective functions to analyze the squares positions and apply the corresponding transform to the source image.

The problem is I actually don't understand which parameters should be used, and what is the order of methods calls in step #5. I'm new to coding and maths, so I would appreciate any help, thank you!

Answer 1

I assume your main question is how to find a transformation from three 2D correspondences and warp an image accordingly.

Note that 3 points only define an affine transformation, not a perspective one. (Affine transformations are perfectly good if your viewing angle is small, for instance if an object is far away from the camera.)

A 2D affine matrix is a 3x3 matrix where the last row is [0 0 1]. (For a perspective matrix, the zero values may be arbitrary.)

You can calculate the affine transformation by solving one linear system, as described in the answer https://stackoverflow.com/a/2756165/2079934. (The solution is unique.)

When you have the affine matrix, you can apply warpAffine (not warpPerspective) to rectify the image. http://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html#warpaffine