Get corners of rectangle from black and white image

Question

Let's say that we have the following black and white image (image.png):

We load the image using OpenCV2 with the following code:

import cv2
img = cv2.imread('image.png')

How can we detect the corners of all white rectangles in an image? We can assume that all rectangles are parallel to the corners of the image.

Result in this case should be in the following form (res[0] is left rectangle and res[1] is right rectangle):

res = [
    [
        (20, 30), # upper left
        (40, 30), # upper right
        (20, 80), # bottom left
        (40, 80)  # bottom right
    ],
    [
        (100, 20), # upper left
        (140, 20), # upper right
        (100, 70), # bottom left
        (140, 70)  # bottom right
    ]
]

Answer 1

There are a few possibilities:

• the "Harris Corner Detector" is good at finding corners - see here
• you can use OpenCV's findContours()
• you could use "Hit-or-Miss" morphology to look for corners
• you could convolve the image with a kernel and look for specific outputs

So, looking at the last option, if we slide a 2x2 pixel kernel over the image and multiply each of the elements underneath the kernel by 1 and add them all together, and then find all the pixels where that total comes to 255, that will be a 2x2 square where exactly one pixel is white - and that is a corner:

import cv2
import numpy as np

# Load image as greyscale
im = cv2.imread('tZHHE.png',cv2.IMREAD_GRAYSCALE)

# Make a 2x2 kernel of ones
kernel = np.ones((2,2), dtype=np.uint8)

# Convolve the image with the kernel
res = cv2.filter2D(im.astype(np.int16), -1, kernel)

# Get coordinates of points where the sum of the 2x2 window comes to 255
corners = np.where(res==255)

Sample Output

(array([101, 101, 118, 118, 166, 166, 174, 174]),
 array([274, 307, 117, 134, 274, 307, 117, 134]))

---

Looking at the "Hit-or-Miss" morphology method, I will do it with ImageMagick straight in the Terminal, but you can equally do it with other Python libraries:

magick tZHHE.png -alpha off -morphology HMT Corners result.png

As always, I am indebted to Anthony Thyssen for his ImageMagick examples as linked above. We are looking for these specific shapes with the "Hit-or-Miss" morphology:

Keywords: Python, OpenCV, image processing, convolution, corner detect, corner detector, corner detection, ImageMagick Hit-or-Miss morphology.

Answer 2

This works for any number of rectangles:

import cv2 as cv
import pprint as pprint

img = cv.imread("tZHHE.png") # read image
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY) # make grayscale image

cv.imshow("Our initial image",img) # show our original image 

corners = cv.goodFeaturesToTrack(gray,2000,0.01,5) # find our corners, 2000 is the number of corners we can detect, 5 is the distance between corners 

xylist = [] #put all of our xy coords in here 

for corn in corners: # extract our corners and put them in xylist
    x,y = corn[0]
    xylist.append((x,y))
    x = int(x)
    y = int(y)
    cv.rectangle(img, (x-2,y-2), (x+2,y+2), (100,100,0),-1) # now mark where our corners are on our original image
            
res = [[] for i in range(int(len(xylist)/4))] # generate i nested lists for our rectangles
        
for index, item in enumerate(xylist): # format coordinates as you want them
   res[index % int(len(xylist)/4)].append(item)
        
print("\n"+"found ",int(len(xylist)/4) ,"rectangles\n") # how many rectangles did we have?       
print(res)
cv.imshow("Corners?", img) # show our new image with rectangle corners marked

Answer 3

Try findContours()

I suggest you to try findContours() with its companion boundingRect().

Here is how you can make it work.

Load the image in grayscale, then pass it to the function findContours().

img = cv2.imread('tZHHe.png', cv2.IMREAD_GRAYSCALE)
contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

Getting the bounding box from contours, it returns x, y coordinates of the top left corner and w, h the width and height of the box:

[cv2.boundingRect(contour) for contour in contours]
#=> [(117, 118, 17, 56), (274, 101, 33, 65)]

---

View in action

Try this maybe in a Jupyter Notebook to see a kind of animation:

def corners_from_bounding_rect(bounding_rect):
    x, y, w, h = bounding_rect
    points = {'top_left': (x, y), 'top_right':(x+w, y), 'bottom-left': (x, y+h), 'bottom-rigth':(x+w, y+h)}
    return points

Retrieve the points from contours using the method defined:

corner_groups = [corners_from_bounding_rect(cv2.boundingRect(cnt)) for cnt in contours]

# [{'top_left': (117, 118),
#   'top_right': (134, 118),
#   'bottom-left': (117, 174),
#   'bottom-rigth': (134, 174)},
#  {'top_left': (274, 101),
#   'top_right': (307, 101),
#   'bottom-left': (274, 166),
#   'bottom-rigth': (307, 166)}]

Then plot the sequence:

pinned_img = img.copy()

for n, corners in enumerate(corner_groups):
    for name, point in corners.items():
        cv2.circle(pinned_img, point, 10, 255)
        plt.title(f'{n}-{name}')
        plt.imshow(pinned_img)
        plt.show()

The first image from the squence:

Answer 4

Try this code

import cv2

img=cv2.imread('tZHHE.png') # Read my Image
imgContours=img.copy() # Copy my Image for Contours
imgCanny=cv2.Canny(imgContours,10,50)  # Image to Edges

contours,hierarchy =cv2.findContours(imgCanny,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE)
rectangles=[]
for cont in reversed(contours):
    area=cv2.contourArea(cont)
    x, y, w, h = cv2.boundingRect(cont)
    rectangles.append([[x,y],[x+w,y],[x+w,y+h],[x,y+h]])
    cv2.imshow("con", imgContours)
    cv2.waitKey(0)

print(rectangles)
cv2.destroyAllWindows()

left to right rectangles

Output:
[[[273, 100], [307, 100], [307, 166], [273, 166]], [[116, 117], [134, 117], [134, 174], [116, 174]]]