Highlighting specific text in an image using python

Question

I want to highlight specific words/sentences in a website screenshot.

Once the screenshot is taken, I extract the text using pytesseract and cv2. That works well and I can get text and data about it.

import pytesseract
import cv2


if __name__ == "__main__":
    img = cv2.imread('test.png')
    img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    result = pytesseract.image_to_data(img, lang='eng', nice=0, output_type=pytesseract.Output.DICT)
    print(result)

Using the results object I can find needed words and sentences.

The question is how to go back to the image and highlight those word?

Should I be looking at other libraries or there is a way to get pixel values and then highlight the text?

Ideally, I would like to get start and end coordinates of each word, how can that be done?

score 7 · Accepted Answer · answered Jan 10 '19 at 09:10

7

You can use pytesseract.image_to_boxes method to get the bounding box position of each character identified in your image. You can also use the method to draw bounding box around some specific characters if you want. Below code draws rectangles around my identified image.

import cv2
import pytesseract
import matplotlib.pyplot as plt

filename = 'sf.png'

# read the image and get the dimensions
img = cv2.imread(filename)
h, w, _ = img.shape # assumes color image

# run tesseract, returning the bounding boxes
boxes = pytesseract.image_to_boxes(img)use
print(pytesseract.image_to_string(img)) #print identified text

# draw the bounding boxes on the image
for b in boxes.splitlines():
    b = b.split()
    cv2.rectangle(img, ((int(b[1]), h - int(b[2]))), ((int(b[3]), h - int(b[4]))), (0, 255, 0), 2)

plt.imshow(img)

answered Jan 10 '19 at 09:10

Muthukrishnan

2,047
2
16
16

Awesome, that is very helpful. I am still trying to find the answer for the structure of the output. Do you have any reference? – Califlower Jan 10 '19 at 18:17
@Califlower Meaning of "structure of output"? Do you mean the output of `pytesseract.image_to_boxes` ? – Muthukrishnan Jan 11 '19 at 14:43
the output of `pytesseract.image_to_boxes` and `pytesseract.image_to_data` I had to glare at it for few hours to figure out the structure and the meaning of the number. Like for example, that`word_num` restarts enumeration on each `line_num` and each `line_num` is restarted from a`block_num`. Still not sure what the`par_num` means. – Califlower Jan 11 '19 at 17:12
1

@Califlower Were you able to highlight specific words back in the image, if so can you help me too? – Yash Soni Nov 16 '20 at 12:45
Hey, I'm actually working on a very similar task, I'm just using interested in a specific text that i managed to create a mask around it... I want to get back the color of the text, but so far i hope i could calculate average color of the characters rectangle and try to compare those values. Thanks for your solution – Nemanja Nov 12 '21 at 07:44

Highlighting specific text in an image using python

1 Answers1

Linked