OPENCV How to complete missing rectangle?

Question

My question is too easy. I have a rectangle like that. I want to fill missing parts of this square. Also I need coordinates of missing parts. How can i do that? Rectangle

I should be like second image after process.Second Image

Answer 1

Here is the summary of this approach:

1. Scan horizontally row by row
2. Scan vertically col by col
3. Prepare a list of potential points
4. Group points by distance
5. Find the bounding box around the sub-grouped points
6. Compute the rectangles and ignore those ones touch the boundaries of the image.

Source Code:

First of all, import the required libraries.

import numpy as np
import cv2

Secondly, define a function that compute the distance between two points and, more importantly, it allows you to decide whether you want to print "too easy" in the terminal.

def too_easy_distance(pt1, pt2, is_print=False):
    if is_print: print ("too easy!")
    return ((pt1[0] - pt2[0]) ** 2 + (pt1[1] - pt2[1]) ** 2) ** 0.5

Thirdly, load image and find out the width and height.

img = cv2.imread("broken_rect.png", 0)
height, width = img.shape[:2]
active_threshold = 25   # adjust this parameter based on the thickness of borders
potential_pts = []

Now, scan the image horizontally row by row.

for i in range(height):
    if len (np.where(img[i:i+1,:]==0)[1] ) < active_threshold:
        continue
    is_start = False
    is_real_start = False
    for j in range(width):
        if img[i,j] == 0 and is_start is False:
            is_start = True
            continue
        # Enter the valid region
        if is_start and is_real_start is False:
            if img[i,j] == 255:
                is_real_start = True
            else:
                is_real_start = False

        if is_real_start:
            if img[i,j] == 255:
                potential_pts.append((j, i)) 
            else:
                is_real_start = False

Then, scan the image vertically column by column.

for j in range(width):
    if len (np.where(img[:, j:j+1]==0)[1] ) < active_threshold:
        continue
    is_start = False
    is_real_start = False
    for i in range(height):
        if img[i,j] == 0 and is_start is False:
            is_start = True
            continue
        # Enter the valid region
        if is_start and is_real_start is False:
            if img[i,j] == 255:
                is_real_start = True
            else:
                is_real_start = False

        if is_real_start:
            if img[i,j] == 255:
                potential_pts.append((j, i))
            else:
                is_real_start = False

Finally, sort the list and display the result image.

grouped_pts = []  #list of list
for pt in potential_pts:
    if len(grouped_pts) == 0:
        grouped_pts.append([pt])  # create a new sublist and add pt to the new sublist
    # enter the grouped_pts loop
    is_found_place = False
    for sublist in grouped_pts:
        for prev_pt in sublist:
            if too_easy_distance(pt, prev_pt) < 2:
                sublist.append(pt) # add to the list
                is_found_place = True
                break
    # if nothing happend
    if is_found_place is False:
        grouped_pts.append([pt]) # create a new sublist and add pt to the new sublist

result_image = cv2.cvtColor(img.copy(), cv2.COLOR_GRAY2RGB)

for sublist in grouped_pts:
    x,y,w,h = cv2.boundingRect(np.array(sublist))
    if x == 0 or y == 0 or x+w == width or y+h == height:
        continue

    rect = ((x, y), (x+w-1, y+h-1))
    cv2.rectangle(result_image, rect[0], rect[1], (0,255,0), -1)
    print ("Rect Found! [Top-Left]: " + str(rect[0]) + " [Bottom-Right]: " + str([rect[1]]) )

cv2.imshow("result_image", result_image)
cv2.waitKey()

Result Image:

Terminal Output:

Rect Found! [Top-Left]: (267, 62) [Bottom-Right]: [(313, 69)]
Rect Found! [Top-Left]: (552, 327) [Bottom-Right]: [(559, 364)]