0

I have an image where I'm trying to export and do transform as rectangle (no matter in base photo). I have done with finding largest contour in image using mask (trained in pytorch). In variable pr_mask I have stored mask from network. After that from that mask I can get extract contours of my image. So from this image: Root image

I can extract contours like this:

gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray,(5,5),5)
canny = cv2.Canny(blur, 30, 80, 3)

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9,9))

flag, thresh = cv2.threshold(canny, 80, 255, cv2.THRESH_BINARY) 
    
kernel = np.ones((10,10),np.uint8)
dilation = cv2.dilate(canny,kernel,iterations = 1)

ret,thresh = cv2.threshold(dilation, 200, 255, 0)

contours, hierarchy = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)

After that I fetch largest contour and then trying to extract contours:

    approx = np.ones((10,10),np.uint8)
    d = 0

    while (len(approx)>4):
        d=d+1
        approx = cv2.approxPolyDP(largestcontour,d, True)

    cv2.drawContours(imagebase, [largestcontour], -1, (0,255,0), 3)
    
    hull = []
    
    hull.append(cv2.convexHull(largestcontour, False))

    cv2.drawContours(imagebase, hull, -1, (0,0,255), 3)

After that I get tthis result: enter image description here

Is it possible to extract square (rectangle) which will transform my image to exact square without rounded corners and to fit whole image?

I'm trying to find a way how to solve this "perspective" issue and align element to specific position. My image is taken by cameras, so it can be in different positions, where boundingRectangle will not work ... any suggestions?

Kristián Stroka
  • 698
  • 1
  • 8
  • 23
  • 2
    You are looking for `cv2.minAreaRect()`, see (this answer](https://stackoverflow.com/questions/18207181/opencv-python-draw-minarearect-rotatedrect-not-implemented). After obtaining the four corners of your rectangle you can align it using a homography matrix – Jeru Luke May 25 '22 at 08:28
  • Maybe you can fit a quadrilateral to that contour, using the `approxPolyDP` function, that way you can get the four corners you need to rectify the perspective via the `warpPerspective` function. – stateMachine May 25 '22 at 08:33
  • well minAreaRect is not correct, since it is not close to red line/green, it is much bigger than just image :/ – Kristián Stroka May 25 '22 at 08:37
  • @stateMachine what do you mean with "quadrilateral to that contour"? – Kristián Stroka May 25 '22 at 08:39
  • 2
    approxPolyDP will completely fail here. given sufficient tolerance, it will maintain one point on each rounded corner, which causes the edges to move completely away where they're supposed to be. it's useless here. -- minarearect won't give you perspective or shear, it'll only be useful for rotation. unfortunately, you'll have to apply some kind of Hough transform, or segment the card's contour into straight and curved sections, line-fit the straight sections (edges of the rectangle) and then calculate intersection points of those lines. – Christoph Rackwitz May 25 '22 at 10:33
  • 1
    with minAreaRect you can estimate the rotation and split your contour into 4 "sides". For each side you can estimate a single best straight line by RANSAC line fitting or hough transform. Afterwards you'll have the 4 corners from intersection of those lines. Might fail however if you dont know the used camera and that camera has a strong wide angle lens, so that straight lines appear curved. There you would need a lens distortion correction anyways. – Micka May 25 '22 at 10:53
  • I'm trying to split that contour without corners, what I get is to create mask with larger squares in minAreaRect corners, and then by mask I split just points which are not part of that squares ... but still I have one contour without corner points, but it is as separate contour. Is it possible to split to 4 separate lines? – Kristián Stroka May 25 '22 at 13:51
  • If you can estimate inverse-warp, do "inverting it" becomes the warp you want? If such way is not wrong, the problem may become to "optimize the inverse-warp". and evaluation of "inverse-warp" may be done as: warping some points on outline of the exact square onto image coordinate (with the "inverse-warp"), then evaluate this inverse-warp based on distance between warped locations and contour edge. (Of course, avoid useing points near by corners) – fana May 26 '22 at 01:50

0 Answers0