2

We print 500 bubble surveys, get them back, and scan them in a giant batch giving us 500 PNG images.

Each image has a slight variations in alignment, but identical size and resolution. We need to register the images so they're all perfectly aligned. (With the next step being semi-automated scoring of the bubbles).

If these were 3D-MRI images, I could accomplish this with a single command line utility; But I'm not seeing any such tool for aligning scanned text documents.

I've played around with opencv as described in Image Alignment (Feature Based) using OpenCV, and it produces dynamite results when it works, but it often fails spectacularly. That approach is looking for documents hidden within natural scenes, a much harder problem than our case where the images are just rotated and translated in 2D, not 3.

I've also explored imreg_dft, which runs consistently but does a very poor job -- presumably the dft approach is better on photographs than text documents.

Does a solution for Image Registration of Scanned Forms already exist? If not, what's the correct approach? Opencv, imreg_dft, or something else?

Similar prior question: How to find blank field on scanned document image

Interesting Integers
  • 21
  • 2
  • if you can afford to provide control points (at 4 corners of the actual image border), then you could try doing a affine transformation like https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/ – pangyuteng Jul 27 '19 at 23:33
  • Printing control points on the documents would have definitely been a great idea, but sadly surveys were printed, filled out, and scanned before I was involved -- can't add 'em now. – Interesting Integers Jul 27 '19 at 23:39
  • Can you add a (blank) example image? – J.D. Jul 27 '19 at 23:55
  • I can't, but fortunately the publisher already has! Images are here: https://www.amazon.com/MacArthur-Bates-Communicative-Development-Inventories-Sentences/dp/1557668892 https://images-na.ssl-images-amazon.com/images/I/51UQs%2Bp8ByL._SX375_BO1,204,203,200_.jpg – Interesting Integers Jul 28 '19 at 00:10
  • So the image has these standardized solid black rectangles that are potentially useful. The bubbles themselves are similarly standardized. I'm just surprised there isn't a simple solution for any text form, regardless of its content. That seems so much easier that registering MRIs – Interesting Integers Jul 28 '19 at 00:16
  • Have you tried cv2.findTransformECC and then cv2.warpAffine as opposed to ORB feature matching? – fmw42 Jul 28 '19 at 00:40
  • I've tried findTransformECC and WarpAffine, but the results are comparable to img_dft -- it DOES run consistently, which is more than I can say for the Orb+Matcher+FindHomography solution. But the [resulting image](https://imgur.com/a/hFPWwpf) is pretty badly misaligned. In some cases the bubbles don't even overlap across the forms. (Note: the large white regions in the image are just my redactions) – Interesting Integers Jul 28 '19 at 01:27

1 Answers1

2

What you can try is using the red outline of the answer boxes to create a mask where you can select the outline. I create a sample below. You can also remove the blue letters by creating a mask for the letters, inverting it, then apply it as a mask. I didn't do that, because he image of the publisher is low-res, and it caused issues. I expect your scans to perform better.

When you have the contours of the boxes you can transform/compare them individually (as the boxes have different sizes). Or you can use the biggest contour to create a transform for the entire document.

You can then use minAreaRect to find the cornerpoints of the contours. Threshold the contourArea to exclude noise / non answer area's.

enter image description here

import cv2
import numpy as np
# load image
img = cv2.imread('Untitled.png')
# convert to hsv colorspace
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# define range of image bachground in HSV
lower_val = np.array([0,0,0])
upper_val = np.array([179,255,237])

# Threshold the HSV image 
mask = cv2.inRange(hsv, lower_val, upper_val)

# find external contours in the mask
contours, hier = cv2.findContours(mask, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
# draw contours
for cnt in contours:
    cv2.drawContours(img,[cnt],0,(0,255,0),3)

# display image
cv2.imshow('Result', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
J.D.
  • 4,511
  • 2
  • 7
  • 20