0

I have a batch of screenshots where I'm printing the coordinates of buttons with certain text (in this case, buttons with the word "exit"). Finding the coordinates of the text was pretty straightforward with EasyOCR. My problem is that I'm not sure how to find the coordinates of the border around the text.

import cv2
import easyocr

img = cv2.imread('C:/Users/xyz/PycharmProjects/findTextImage_v0/testBank/sampleScreen.png')

text = easyocr.Reader(['en'], gpu = True)
result = text.readtext(img)

for item in result:
    if "EXIT" in item:
        print(item[0], item[1])
    else:
        print("False")

A sample screenshot is below for context. Unfortunately, the buttons are not consistent between screenshots. Sometimes, the margins are bigger than the one below, the button is in a different place, the button's background color is a lighter gray, etc.

Previously, I've tackled this by using template matching to find the button, but that's not always accurate and involves a fair bit of manual tweaking per project and multiple image templates and functions.

Once I find the specific text I'm looking for in an image, how can I go about finding its nearest borders and its coordinates?

Thanks for any help!

enter image description here

mvCode
  • 25
  • 5
  • do you just want the center of the button? the center of the recognized text is close enough. – Christoph Rackwitz Jan 18 '23 at 22:47
  • 1
    No, I need the top left position and the bottom right position of the button (black border) itself. – mvCode Jan 18 '23 at 23:15
  • in that case, I'd (2) find all "boxes", in tree order (they hopefully don't "intersect" but contain each other), and then (2) assign the text to the smallest enclosing box. -- finding boxes could be accomplished in various ways. if you can rely on buttons always having a black border, that'd work. if they always have the same gray body, that'd work too. – Christoph Rackwitz Jan 19 '23 at 09:36
  • I think that might be too specific for me to re-use this code for other buttons. Ideally I'd find a way to search for other text buttons for other uses. I think I need to find a way to find the closest border to the left of Coordinate A and get that X&Y value. Then find the closest border to the right of Coordinate B and get that X&Y value. I just have no idea where to even start. My attempts at finding all boxes on my sample images haven't given the expected results. – mvCode Jan 19 '23 at 18:27
  • followup: https://stackoverflow.com/questions/75186033/selecting-area-of-image-based-on-color-similar-to-gimp-photoshops-magic-wand-t – Christoph Rackwitz Jan 20 '23 at 16:12
  • Thank you - I wasn't sure how fine a line I was walking regarding spam/follow-up. Your comments have been very helpful in getting me to re-think and pivot to a different solution. – mvCode Jan 20 '23 at 16:16
  • just giving people some context to know where you're going with this – Christoph Rackwitz Jan 20 '23 at 23:28

0 Answers0