0

I am on ubuntu.

I have a pdf file with pages divided into a grid. Each block of the grid contains name/age/dob/photo of a candidate. some records have a watermark "disqualified"

I need to scrape his pdf, with disqualified candidates in a separate list. Using pyPdf I was able to get individual records, but it also includes watermarked candidates.

How to detect the watermark? If I can get the coordinates of the watermark, how do I match it with the candidate?

I am open to solutions other than python pyPdf

sulabh
  • 1,097
  • 1
  • 8
  • 22
  • 1
    Most likely a solution will depend on the very structure of pdfs your have. Could you, therefore, supply a sample? – mkl Jun 09 '13 at 20:14
  • Here is the link to pdf sample https://docs.google.com/file/d/0BzbpFYSySNBZNnJIUXk0VFdweFE/edit?usp=sharing – sulabh Jun 12 '13 at 12:00
  • Also using pdftops on ubuntu i got the ps file as https://docs.google.com/file/d/0BzbpFYSySNBZYmVKN25uU1JxTGs/edit?usp=sharing – sulabh Jun 12 '13 at 12:02

1 Answers1

1

(Actually this is not an answer but merely an analysis to bit for a comment.)

I don't know pyPdf (or any python PDF classes) myself, but here is how the watermark is created for a sample entry; based upon this, anyone knowing pyPDF well enough, may more easily advice.

The Roundup

Depending on how pyPDF (or other python PDF classes) allows access to the page content, there are two major basic approaches:

  1. If the class returns information on content (text and image) in their order in the page content stream: The watermark image xobject is referred to right before the data of the entry. Thus, any entry preceded by the drawing of a xobject image is marked.

  2. If otherwise the information are not given in the order indicated by the page content stream, coordinate comparison must be used which per se is quite straight forward. In that case it might be of interest that the images are inserted with a [0.1 0 0 0.1 0 0] transformation matrix in action while the text is drawn with an identity transformation matrix.

The Details

This is entry # 200; the other watermarked entry is constructed similarly:

dataset 200 with DELETED watermark

Watermarking is done by means of an image xobject. There is but one image xobject defined for the page used by both watermarked entries:

4 0 obj
<</Type/Page/MediaBox [0 0 595 841]
/Rotate 0/Parent 3 0 R
/Resources<</ProcSet[/PDF /ImageC /ImageI /Text]
    /ColorSpace 18 0 R
    /ExtGState 19 0 R
    /XObject 20 0 R
    /Font 21 0 R
    >>
/Contents 5 0 R
>>
endobj 
20 0 obj
<</R17
17 0 R>>
endobj
17 0 obj
<</Subtype/Image
/ColorSpace 16 0 R
/Width 128
/Height 88
/BitsPerComponent 8
/Filter/FlateDecode/Length 463>>stream 
[...]
endstream
endobj

In the content stream this xobject /R17 is inserted right before the data of the entry itself is drawn:

q 0.1 0 0 0.1 0 0 cm
[...]
q 1045 0 0 495 462.5 6510.5 cm
/R17 Do
Q
q
10 0 0 10 0 0 cm BT
0.000487366 Tc
/R10 8 Tf
1 0 0 1 86 650.75 Tm
(Sex : Male)Tj
0.000304794 Tc
-64 0 Td
(Age : 43)Tj
-0.000140686 Tc
-1 11.05 Td
(House No :)Tj
-0.00002085 Tc
1 31.95 Td
(Name :)Tj
0.00008575 Tc
/R12 7.15 Tf
25.5 17.8 Td
( 200 )Tj
ET
Q
1547.5 6475 485 535.5 re
S
q
10 0 0 10 0 0 cm BT
-0.000403137 Tc
/R14 8 Tf
1 0 0 1 145.1 708.5 Tm
(XVX0001081)Tj
0.000421651 Tc
/R14 7.05 Tf
-90.35 -14.95 Td
(Ramesh Kumar)Tj
0.000373332 Tc
/R10 7.05 Tf
-33 -12.75 Td
(Father's )Tj
0.000193787 Tc
7.3 TL
(Name)'
0.00037774 Tc
/R14 7.05 Tf
40.25 1.8 Td
(Ram Singh)Tj
0 Tc
2.5 -11.85 Td
(37)Tj
0.00137196 Tc
/R12 7.15 Tf
-5.25 13.35 Td
(:)Tj
mkl
  • 90,588
  • 15
  • 125
  • 265