Improving OCR performance on multi-paragraph scans

Question

I'm working on a project that involves extracting text scientific papers stored in PDF format. For most papers, this is accomplished quite easily using PDFMiner, but some older papers store their text as large images. In essence, a paper is scanned and that image file (typically PNG or JPEG) comprises the entire page.

I tried using the Tesseract engine through it's python-tesseract bindings, but the results are quite disappointing.

Before diving into the questions I have with this library, I would like to mention that I'm open to suggestions for OCR libraries. There seem to be few native python solutions.

Here is one such image (JPEG) on which I am trying to extract text. I the exact code provided in the example snippets on the python-tesseract google code page I linked to above. I should mention that the documentation is a bit sparse, so it's quite possible that one of the many options in my code is misconfigured. Any advice (or links to in-depth tutorials) would be much appreciated.

Here is the output from my attempt at OCR.

My questions are as follows:

1. Is there anything suboptimal in the code I'm using? Is there a better way of doing this? A different library perhaps?
2. What kind of preprocessing can I perform to improve detection? The images are all B&W, but should I perhaps set a threshold and set anything above it to a single-value black color and everything below it to a null-value white color? Anything else?
3. A more specific question: can performance be improved by performing OCR on single words? If so, can anyone suggest a way of delimiting single words in an image file (e.g.: the one linked above) and extracting them into separate images which can be treated independently?
4. Can the presence of graphs and other images embedded in the PDF page image interfere with OCR? Should I remove these? If so, can anyone suggest a method for removing them automatically?

EDIT: For simplicity, here is the code I used.

import tesseract
api = tesseract.TessBaseAPI()
api.Init(".","eng",tesseract.OEM_DEFAULT)
api.SetPageSegMode(tesseract.PSM_AUTO)

mImgFile = "eurotext.jpg"
mBuffer=open(mImgFile,"rb").read()
result = tesseract.ProcessPagesBuffer(mBuffer,len(mBuffer),api)
print "result(ProcessPagesBuffer)=",result

And here is the alterative code (whose results are not shown in this question, although the performance appears to be quite similar).

import cv2.cv as cv
import tesseract

api = tesseract.TessBaseAPI()
api.Init(".","eng",tesseract.OEM_DEFAULT)
api.SetPageSegMode(tesseract.PSM_AUTO)

image=cv.LoadImage("eurotext.jpg", cv.CV_LOAD_IMAGE_GRAYSCALE)
tesseract.SetCvImage(image,api)
text=api.GetUTF8Text()
conf=api.MeanTextConf()

Could anyone explain the differences between these two snippets?

Answer 1

Tesseract is very good on clean input text (like your example) if you tinker a bit. some suggestions:

• Before automating, start with tesseract at the command line
• Restrict your character set if possible (e.g. take a look in /usr/local/share/tessdata/configs at ./digits - configure it for English characters upper/lower case etc) and provide it as a command line argument
• Only use PNG or TIFF images (TIFF for older versions) as JPG introduces artefacts
• Upsample the image so your text is larger than the current tiny font. Tesseract lines >10 pixel high characters (if memory serves), it certainly performs worse with tiny characters
• No need to do thresholding if you're bi-level already but it won't hurt if you do and you can see exactly the same image that tesseract will see

I'll check back here to see if I can help more but do join the tesseract mailing list, they're really helpful.

Sidenote - I have some patches for pytesseract which I ought to publish for getting characters & confidences & words via the API (which wasn't possible a couple of months back). Shout if they might be useful.

Answer 2

The first example reads the file as a buffer and then relay it to tesseract-ocr without doing any modification while the second one reads file into opencv format which will then allow you to do some image touch up like changing the aspect ratio, gray scale and etc using the cv library. The second method is very useful if u want to do the image manipulation before passing the image to tesseract.

BTW, I am the owner of python-tesseract. If u want to ask question, you could always welcome to forward your question to http://code.google.com/p/python-tesseract

Joe