OpenCV: Segment each digit from the given image. Digits are written in each cell of a row matrix. Each cell is bounded by margins

Question

I have been trying to recognise handwritten letters (digits/alphabet) from a form-document. As it is known that form-documents have 1d row cells, where the applicant has to fill their information within those bounded cells. However, I'm unable to segment the digits(currently my input consists only digits) from the bounding boxes.

I went through the following steps:

1. Reading the image (as a grayscale image) via "imread" method of opencv2. Initial Image size:19 x 209(in pixels).

pic = "crop/cropped000.jpg"
newImg = cv2.imread(pic, 0)

2. Resizing the image 200% its original size via "resize" method of opencv2. I used INTER_AREA Interpolation. Resized Image size: 38 x 418(in pixels)

h,w = newImg.shape
resizedImg = cv2.resize(newImg, (2*w,2*h), interpolation=cv2.INTER_AREA)

3. Applied Canny edge detection.

v = np.median(resizedImg)
sigma = 0.33
lower = int(max(0, (1.0 - sigma) * v))
upper = int(min(255, (1.0 + sigma) * v))
edgedImg = cv2.Canny(resizedImg, lower, upper)

4. Cropped the contours and saved them as images in 'BB' directory.

im2, contours, hierarchy = cv2.findContours(edgedImg.copy(),cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)

num = 0
for c in contours:
    x, y, w, h = cv2.boundingRect(c)
    num += 1
    new_img = resizedImg[y:y+h, x:x+w]
    cv2.imwrite('BB/'+str(num).zfill(3) + '.jpg', new_img)

Entire code in summary:

pic = "crop/cropped000.jpg"
newImg = cv2.imread(pic, 0)

h,w = newImg.shape
print(newImg.shape)
resizedImg = cv2.resize(newImg, (2*w,2*h), interpolation=cv2.INTER_AREA)
print(resizedImg.shape)

v = np.median(resizedImg)
sigma = 0.33
lower = int(max(0, (1.0 - sigma) * v))
upper = int(min(255, (1.0 + sigma) * v))
edgedImg = cv2.Canny(resizedImg, lower, upper)

im2, contours, hierarchy = cv2.findContours(edgedImg.copy(),cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)

num = 0
for c in contours:
    x, y, w, h = cv2.boundingRect(c)
    num += 1
    new_img = resizedImg[y:y+h, x:x+w]
    cv2.imwrite('BB/'+str(num).zfill(3) + '.jpg', new_img)

Images produced are posted here: https://i.stack.imgur.com/9To5D.jpg

I had to double the image size because Canny edge detection was producing double-edges for an object (However, it still does). I have also played with other openCV functionalities like Thresholding, Gaussian Blur, Dilate, Erode but all in vain.

Answer 1

# we need one more parameter for Date cell width : as this could be different for diff bank
def crop_image_data_from_date_field(image, new_start_h, new_end_h, new_start_w, new_end_w, cell_width):
    #for date each cell has same height and width : here width: 25 px so cord will be changed based on width
    cropped_image_list = []
    starting_width = new_start_w
    for i in range(1,9): # as date has only 8 fields: DD/MM/YYYY

        cropped_img = image[new_start_h:new_end_h, new_start_w + 1 :new_start_w+22]
        new_start_w = starting_width + (i*cell_width)

    cropped_img = cv2.resize(cropped_img, (28, 28))
    image_name = 'cropped_date/cropped_'+ str(i) + '.png'
    cv2.imwrite(image_name, cropped_img)
    cropped_image_list.append(image_name)

# print('cropped_image_list : ',cropped_image_list,len(cropped_image_list))
# rec_value = handwritten_digit_recog.recog_digits(cropped_image_list)
recvd_value = custom_predict.predict_digit(cropped_image_list)
# print('recvd val : ',recvd_value)
return recvd_value

you need to specify each cell width and it's x,y,w,h. I think this will help you.