Extracting selected text by bounding box from an image

Question

I am trying to fetch selected text by bounding box on an Image. like if only on word is selected by bounding box and I want to fetch that text and convert it into the text file. Please see my code and give some review so I can implement that functionality.

So far what I've done I've converted the PDF file to image with bounding box over the text.

import numpy as np
import csv
import io
from PIL import Image
import pytesseract
from wand.image import Image as wi
from pytesseract import Output
import cv2

pdf = wi(filename="samplecompany.pdf", resolution=100)
pdfImg = pdf.convert('jpg')
j = 1
for img in pdfImg.sequence:
    page = wi(image=img)
    page.save(filename=str(j)+".jpg")
    img1 = cv2.imread(str(j)+".jpg")

    d = pytesseract.image_to_data(img1, output_type=Output.DICT)
    n_boxes = len(d['level'])
    print(n_boxes)
    for i in range(n_boxes):
        (x, y, w, h) = (d['left'][i], d['top']
                        [i], d['width'][i], d['height'][i])
        print((x, y, w, h))
        cv2.rectangle(img1, (x, y), (x + w, y + h), (0, 255, 0), 2)

    cv2.imwrite(str(j)+".jpg", img1)

    cv2.waitKey(0)
    j += 1

this code is working fine I need to fetch desired text from images which I've created.using bounding box location

can give some image example? it is hard to guess what you are seeing. Give input, what you have now, and what you want it to be in drawing. Then we will give you answer — Dr Yuan Shenghai, Jun 04 '19 at 14:14
I've added the image in question section please see it,what I've generated I just want to converted into the text (Only selected text by the box) — Neeraj Nawariya, Jun 05 '19 at 11:39

score 3 · Accepted Answer · answered Jun 20 '19 at 09:28

You can use this code to get custom text from a an image and change and modify accordingly and this is also save your text to an text file

import io
import cv2
import numpy as np
import pytesseract
from PIL import Image
from pytesseract import Output
from wand.image import Image as wi
import sys


pdf = wi(filename="Resume.pdf", resolution=100)
pdfImg = pdf.convert('jpg')
j = 1
imgBlobs = []
img1= []
for img in pdfImg.sequence:
    page = wi(image=img)
    page.save(filename=str(j)+".jpg")
    img1.append(cv2.imread(str(j)+".jpg"))
    j += 1

extracted_text = []

for img2 in img1:
    d = pytesseract.image_to_data(img2, output_type=Output.DICT)
    n_boxes = len(d['level'])
    print(n_boxes)
    extracted_text.append(d['text'][9])
    (x, y, w, h) = (d['left'][9], d['top'][9], d['width'][9], d['height'][9])
    cv2.rectangle(img2, (x, y), (x + w, y + h), (0, 255, 0), 2)


    cv2.imshow('img', img2)

    print(d)


with open('Prototype.txt', 'w') as filehandle:
        for listitem in extracted_text:
            filehandle.write('%s\n' % listitem)

This is for getting all the OCR text from this image right. I Think the question is to get only the highlighted text using bounding box — Prajeesh T S, Feb 05 '21 at 04:34
@PrajeeshTS thats what my code doing . you see this line extracted_text.append(d['text'][9]) (x, y, w, h) = (d['left'][9], d['top'][9], d['width'][9], d['height'][9]) I'm passig the value hardcoded value not the range in i so only bounded box text is extracted — Rajeev Shankhwar, Feb 16 '21 at 05:42
Can we draw multiple rectangles with help of mouse and extract the required information? — Nithin Reddy, Nov 22 '21 at 07:42

score 0 · Answer 2 · answered Jun 05 '19 at 11:45

0

let r denotes the bounding box (x, y, w, h) of your target. the croping can be done by

# Crop image
Croped_image = Image[int(r[1]):int(r[1]+r[3]), int(r[0]):int(r[0]+r[2])]

So you have the bounding box. Then try to detect the text from the segmented croped image. Whatever side is what you wanted

answered Jun 05 '19 at 11:45

Dr Yuan Shenghai

1,849
1
6
19

Can we draw multiple rectangles with help of mouse and extract the required information? – Nithin Reddy Nov 22 '21 at 07:42
possible. use the mouse click down event as bounding box 1 top left init (x1,y1) then release event as box 1 bottom right (x2,y2). Repeat click event. then you can do multiple – Dr Yuan Shenghai Nov 25 '21 at 07:49

Extracting selected text by bounding box from an image

2 Answers2

Linked