How to find the contour of a completed form scanned image?

Question

I would like to detect the contour of the completed form in this scan.

Ideally I would want to find the corners of the table painted with red.

My final goal is to detect that the whole document was scanned and that the four corners are within the boundaries of the scan.

I used OpenCV from python - but it was not able to find the contour of the big container.

Any ideas?

Sure, I tried this solution for contour detection:https://github.com/Breta01/handwriting-ocr/blob/master/notebooks/page_detection.ipynb , also described here :https://bretahajek.com/2017/01/scanning-documents-photos-opencv/. I also tried some scripts for finding contours, described here: https://medium.com/analytics-vidhya/how-to-detect-tables-in-images-using-opencv-and-python-6a0f15e560c3. When running the first solution, it returns the contour of the picture (I guess because there is no contrast between the scanned page and background). For the second, I could not find the greatest poly. — Corovei Andrei, Feb 04 '20 at 10:54
Looks like you're trying to detect the table by its contour. Check my answer here https://stackoverflow.com/questions/57210148/what-is-the-best-way-to-extract-text-contained-within-a-table-in-a-pdf-using-pyt/57664735#57664735 — Dmytro, Feb 04 '20 at 12:43

score 2 · Accepted Answer · answered Feb 05 '20 at 00:35

With the observation that the form can be identified using the table grid, here's a simple approach:

Obtain binary image. Load the image, grayscale, Gaussian blur, then Otsu's threshold to get a binary image
Find horizontal sections. We create a horizontal shaped kernel and find horizontal table lines and draw onto a mask
Find vertical sections. We create a vertical shaped kernel and find vertical table lines and draw onto a mask
Fill text document body and morph open. We perform morph operations to close the table then find contours and fill the mask to obtain a contour of the shape. This step fulfills your needs since you can just find contours on the mask but we can go further and extract only the desired sections.
Perform four-point perspective transform. We find contours, sort for the largest contour, sort using contour approximation then perform a four-point perspective transform to obtain a birds eye view of the image.

Here's the results:

Input image

Detected contour to extract highlighted in green

Output after 4-point perspective transform

Code

import cv2
import numpy as np
from imutils.perspective import four_point_transform

# Load image, create mask, grayscale, and Otsu's threshold
image = cv2.imread('1.jpg')
mask = np.zeros(image.shape, dtype=np.uint8)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3,3), 0)
thresh = cv2.adaptiveThreshold(blur,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV,11,3)

# Find horizontal sections and draw on mask 
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (80,1))
detect_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(mask, [c], -1, (255,255,255), -1)

# Find vertical sections and draw on mask 
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,50))
detect_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(mask, [c], -1, (255,255,255), -1)

# Fill text document body
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
close_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9,9))
close = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, close_kernel, iterations=3)
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(mask, [c], -1, 255, -1)

# Perform morph operations to remove noise
# Find contours and sort for largest contour
opening = cv2.morphologyEx(mask, cv2.MORPH_OPEN, close_kernel, iterations=5)
cnts = cv2.findContours(opening, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cnts = sorted(cnts, key=cv2.contourArea, reverse=True)
displayCnt = None

for c in cnts:
    # Perform contour approximation
    peri = cv2.arcLength(c, True)
    approx = cv2.approxPolyDP(c, 0.02 * peri, True)
    if len(approx) == 4:
        displayCnt = approx
        break

# Obtain birds' eye view of image
warped = four_point_transform(image, displayCnt.reshape(4, 2))

cv2.imwrite('mask.png', mask)
cv2.imwrite('thresh.png', thresh)
cv2.imwrite('warped.png', warped)
cv2.imwrite('opening.png', opening)

score 0 · Answer 2 · answered Feb 04 '20 at 12:45

What about using the Hough transform with a narrow direction range, to find the verticals and horizontals ? If you are lucky, those that you need will be the longest, and after selecting them you can reconstruct the rectangle.

How to find the contour of a completed form scanned image?

2 Answers2