6

I am doing a project in opencv to detect handwritten characters from a user filled form. I have made algorithm to detect the skew angle of the scanned image using Hough Line Transform. But it does not work when the image is 180 degree rotated since 0 and 180 degree are treated as same by Hough Line function. My image contains some rectangles to fill data in them and some text. So how do i detect if a scanned image is 180 degree rotated or not?

Since I will have to first correct the skew angle of the image then only I can detect exactly where on the image user filled data (which I need to extract) lies using rectangle coordinates from the empty template form provided earlier, answers without using chacater recognition are appreciated.

g.alisha12
  • 246
  • 1
  • 3
  • 10

5 Answers5

3

To lift the 180° degrees ambiguity, only OCR can tell you: perform two reads on the deskewed text, one using the given angle, the other one using the angle + 180°, and keep the most successful read.

Unless you have some a priori information it's the only way, as other image processing operations don't know about characters.

UPDATE:

Some strings are forever ambiguous, like 0689HINOSXZ <=> ZXSONIH6890.

If the layout of the text is known (boxes) and asymmetric, it is a relatively easy matter to check matching of the text strings to the layout: choose a box (such as the topmost) and a string (the topmost), and align them by translation; then see how the other boxes and strings match (using a nearest neighbor rule) and establish the correspondences. Compare results with the straight and flipped layout, and keep the best overall area of overlap.

For reliability, it can be better to try more than a starting box/string pair, as there can be some ambiguity to which is the topmost (it could even be missing).

0

Isn't your problem more general? Let's say, you detect a skew angle of +45 degrees and rotate the image by -45 degrees. Then it could still be that the image is rotated by 180 degrees because it was not rotated +45 degrees but -135 instead.

Anyway, to the actual question: I am not an expert in character recognition but I think if you use it anyway in your application, couldn't you just try character recognition for both rotations and then choose the one that gets stronger response?

Severin
  • 668
  • 2
  • 7
  • 19
  • This is where the problem lies. After rotation by -45 degrees I want to see whether the new image is now 0 degree or 180 degrees rotated. In the initial stage I dont know where the characters in the image lie. It can be any random image. So character recognition cannot be applied. – g.alisha12 Aug 08 '14 at 19:14
  • Oh, in this case I read your question the wrong way :) Well I don't quite understand. I thought you want to detect characters, so you will have to find them anyway, right? :) So my suggestion is to just rotate the image such that lines lie horizontally (with Hough transform as you described) and to find out if you have to rotate it by 180 degrees, you try character recognition on both orientations (0 and 180 degree rotated). – Severin Aug 08 '14 at 19:45
0

If you match the rectangles in your template with those of the skew corrected image, you'll be able to get the correct orientation (but only if there's no symmetry in the placement of those rectangles). For matching you may be able to use the rectangles in your template as a mask to extract regions from skew corrected image.

EDIT

Suppose your template and the skew corrected image look like this (in the best case where there are no displacements in skew corrected) :

enter image description here enter image description here

Then you can use the template as a mask to copy data from skew corrected image. Then check what fraction of the white pixels in the template is contained in the copied image. This value will be very low for a 180 degree rotated image.

But as you say, this won't work in practice because of the displacements. Then may be you can try template matching (cross correlation) in which you use the template image as the template. Location of the strongest peak and the strength would give you some indication of the orientation. You can perform template matching at a reduced resolution so it runs faster.

dhanushka
  • 10,492
  • 2
  • 37
  • 47
  • Yes I have the coordinates of rectangles in the template image and even if I extract the same area from the new image, how will I find that the area contains the same rectangle? Problem is that they are filled now so neither contour detection not vertical projection techniques can be applied. Moreover, even after skew correction of the image, the new image can be vertically or horizontally displaced. So, I dont think matching rectangles is the best idea. – g.alisha12 Aug 11 '14 at 17:17
0

You could try to match keypoints (Harris, Sift, ...) from the scanned image and the empty template. With the matched points you can easily find a transformation to align the scanned image with the template. This may work for your case, but you are more likely to succeed if the are some textured logos in the images, as it's usually the case for forms.

juanmanpr
  • 21
  • 5
0

Can't you simple compute two cross-correlations? One with 180 rotation and one without? The one with the matching rectangle should give you a higher correlation maximum (provided the image contrast of the remaining page is not too misleading, but some pre-filtering could help here.)

BmyGuest
  • 6,331
  • 1
  • 21
  • 35