5

I have been battling with Google and the limited documentation of PDFMiner for the last several hours, and although I feel close, I'm just not getting what I need. I've worked through http://www.unixuser.org/~euske/python/pdfminer/ and all three of the YouTube videos to gain a better understanding about PDFs and I'm able to output raw text just fine.

I am working on a script to parse multiple PDF pages. Unfortunately, for this project I am dealing with poor quality PDF files, and the only reliable constant I see is the physical location of text strings being exactly the same. Although I've read hints that text strings can be extracted by physical coords, I have yet to see a working example.

Is there anyone out there who could shed some light on how this is done with PDFMiner? I am open to other modules if there is an obvious better choice, however I need to stick with Python for the script.

Additionally, I have tried PyPdf to no success as well (other than basic text output).

Thanks!

user1145643
  • 896
  • 5
  • 15
  • 26

2 Answers2

9

I've been writing a library to try to simplify this process, pdfquery. To extract text from a particular place in a particular page, you would do:

pdf = pdfquery.PDFQuery(file)
# load first, third, fourth pages
pdf.load(0, 2, 3) 
# find text between 100 and 300 points from left bottom corner of first page
text = pdf.pq('LTPage[page_index=0] :in_bbox("100,100,300,300")').text() 
# save tree as XML to try to figure out why the last line didn't work the way you expected :)
pdf.tree.write(filename, pretty_print=True)

If you want to find individual characters within that box, instead of text lines entirely within that box, pass merge_tags=None to PDFQuery (by default it merges consecutive characters into a single element to make the tree less ridiculous, so the whole line would have to be inside the box). If you want to find anything that partially overlaps the box, use :overlaps_bbox instead of :in_bbox.

This is basically using PyQuery selector syntax to grab text from a PDFMiner layout, so if your document is too messy for PDFMiner, it may be too messy for this as well, but at least it will be faster to play with.

Jack Cushman
  • 2,110
  • 22
  • 15
3

I was able to find my way around pdfminer thanks to some code by Denis Papathanasiou. The code is discussed in his blog, and you can find the source here: layout_scanner.py

In particular, take a look at the method parse_lt_objs( ). In the final loop, k should be a pair containing the coordinates of that bit of text (and it is discarded). I don't have a working coordinate extractor to post here (I was not interested in them), but it sounds like you'll have no trouble finding your way from there.

Good luck with it!

alexis
  • 48,685
  • 16
  • 101
  • 161
  • Thanks for the link! I was able to pull a nice sorted hash table off that last loop. I wish the quality was better on the doc though. Seems like PDF's are horrid in terms of accurate string extraction; especially when they've been converted between containing text version / image version a few times. If you have any better suggestions, I'm all ears, otherwise I think this is a dead end. – user1145643 Feb 19 '12 at 20:32
  • I'm afraid I don't know of another tool. PDF is just too close to a canvas-level format, who knew. If your problem is detecting word boundaries, I'd suggest trying to model the canvas yourself-- estimate the average letter width, and see when a jump in the x coordinate signifies a space. The only other thought that comes to mind is to look for a tool to convert (usefully) PDF into tagged PDF. Perhaps Adobe provides something, but it might fail on your files too. – alexis Feb 20 '12 at 12:05