0

I'm using pytesseract and opencv to be able to detect numbers fairly real time off of a game. However, it is very inconsistent.

I've changed around a lot of settings and refined the image a lot, yet 90 percent of the time it still doesn't work, even if the image is the same as before.

Here is an example of an image i'm trying to use: enter image description here Here is my code:

from PIL import ImageGrab, Image
import cv2
import time
import argparse
import os
import pytesseract
from ctypes import windll
user32 = windll.user32
user32.SetProcessDPIAware()
pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"
def process_img(screen):
    global filename
    ap = argparse.ArgumentParser()
    ap.add_argument("-p", "--preprocess", type=str, default="thresh",
                    help="type of preprocessing to be done")
    args = vars(ap.parse_args())
    b = screen.copy()

    resizedimg = cv2.resize(b, (750,500))
    ret, thresh = cv2.threshold(resizedimg, 127, 255, cv2.THRESH_BINARY)
    kernel = np.ones((3, 3), np.uint8)
    img = cv2.erode(thresh, kernel, iterations=1)

    img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    if args["preprocess"] == "thresh":
        img_gray = cv2.threshold(img_gray, 0, 255,
                             cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
    elif args["preprocess"] == "blur":
        img_gray = cv2.medianBlur(img_gray, 3)
    blurred = cv2.GaussianBlur(img_gray, (5, 5), 0)
    edged = cv2.Canny(blurred, 50, 200, 255)
    filename = "{}.png".format(os.getpid())
    cv2.imwrite(filename, edged)
    return edged
prevtext = int(100)
while True:
    screen = np.array(ImageGrab.grab(bbox=(790, 1315, 839, 1345)))
    # print('Frame took {} seconds'.format(time.time()-last_time))
    last_time = time.time()
    new_screen = process_img(screen)
    cv2.imshow('window', new_screen)
    if cv2.waitKey(25) & 0xFF == ord('q'):
        cv2.destroyAllWindows()
        break
    time.sleep(.5)
    text = pytesseract.image_to_string(Image.open(filename), lang='eng',
                                       config='--psm 13 --oem 3 -c tessedit_char_whitelist=0123456789')
    os.remove(filename)
    print(text)
    try:
        int(text)
    except:
        text = prevtext

    print("health is " + str(text))
    print("previous health was " + str(prevtext))
    if int(prevtext) > int(text):
        print("Got hit")
    prevtext = text

Is there someway to make this much better (At least 75% success rate)?

Bob Stone
  • 90
  • 1
  • 13
  • As provided, the issue is not reproducible. The apparent image being processed, is `screen`. Edit the question and adjust the code so it uses the file provided. This is not currently a [Minimal, Reproducible Example](https://stackoverflow.com/help/minimal-reproducible-example). – Trenton McKinney Nov 09 '19 at 20:21
  • Running this script as is, causes a `SystemExit` error from `argparse`. `ap.add_argument("-p", "--preprocess", type=str, default="thresh", help="type of preprocessing to be done")` and `args = vars(ap.parse_args())` – Trenton McKinney Nov 09 '19 at 20:34
  • This script actually screen records the screen and creates images (pngs as “frames”) so you should be able to run it and not have an error. However, the picture I have posted is post processing which I have shown in the code before it is saved. Can you change the settings for the actual image to text conversion with pytesseract? – Bob Stone Nov 12 '19 at 20:55

0 Answers0