2

I am trying to get a number from a program on my screen using OpenCV. I have the exact spot on the screen that the value is at and I am able to use image to text recognition to turn the live updating image into text. the text I get is a string as follows (profit: 12.34, whatever it is at the time) I am for some reason unable to just get the number so I am using re.findall to get the number in the string. it works like a hot damn as long as the value is >= 0 . I get a returned value as a float. works perfect. but the second the number goes negative I get this error message

  File "C:/Users/austi/.spyder-py3/OpenCV_Files/Closest_workingCV.py", line 55, in <module>
    price = re.findall("\d+", text)[0]

IndexError: list index out of range

here is my code thus far



import numpy as np
from PIL import ImageGrab
import cv2
import time
import pytesseract
import re



pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"

while True:
    def process_img(original_image):
        processed_img = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
        return processed_img

    def process_img1(original_image1):
        processed_img1 = cv2.cvtColor(original_image1, cv2.COLOR_BGR2GRAY)
        processed_img1 = cv2.Canny(processed_img1, threshold1=200, threshold2=300)
        return processed_img1


    coor1 = (20, 150, 950, 950)
    coor2 = (60, 550, 250, 590)
#    coor3 = (20, 150, 950, 950)


    #last_time = time.time()
    for i in range(2):
        if i == 0:
            x = coor1
            screen = np.array(ImageGrab.grab(bbox=(x)))
            new_screen = process_img(screen)
            #screen('Loop took {} seconds'.format(time.time()-last_time))
            #last_time = time.time() 
            cv2.imshow('window', new_screen)  
            if cv2.waitKey(25) & 0xFF == ord('q'):
                cv2.destroyAllWindows()
                break
        elif  i == 1:
            x = coor2
            screen1 = np.array(ImageGrab.grab(bbox=(x)))
            new_screen1 = process_img(screen1)
            cv2.imshow('window1', new_screen1)   
            text = pytesseract.image_to_string(new_screen1)
            #price = text.split(":")[1]
            price = re.findall("\d+", text)[0]
            #rint(repr(text))
            #price = re.findall("\d+","Foo -111 Bar 55", text)
            price = float(price)
            #text = [float(s) for s in re.findall(r'-?\d+\.?\d*', text)]
            #print(text)
            print(price)
            if cv2.waitKey(25) & 0xFF == ord('q'):
                cv2.destroyAllWindows()
                break         

like I said it updates that number multiple times a second flawlessly so long as its not a negative number. anyone have any suggestions on how to fix this? or any better ways to get to my end goal. I've searched and found similar issues but when I implement any proposed solutions that I have found it either flat out does not work for any number or I have the same issue of only numbers 0 or greater. not sure what to do please help. i included the snippet of my screen that the code is looking at Thanksenter image description here

nathancy
  • 42,661
  • 14
  • 115
  • 137
oz.vegas
  • 129
  • 7
  • Can you post a nonworking example? `re.findall("\d+","Foo -111 Bar 55")` works fine. You could do `r"-?\d+"` if you want the negative sign included. – tdelaney Feb 17 '20 at 23:12
  • Also, its a good idea to wrap the `re.findall` in an exception handler if you want to accept text with no digits at all. – tdelaney Feb 17 '20 at 23:13
  • what do you mena non working example? the code above works for positive numbers but not negative. – oz.vegas Feb 17 '20 at 23:14
  • price = re.findall("\f+","Foo -111 Bar 55", text) is that the right implementation? it returns this 'TypeError: unsupported operand type(s) for &: 'str' and 'int'' – oz.vegas Feb 17 '20 at 23:15
  • Exactly! Can you post some text where it doesn't work? It works fine for me. – tdelaney Feb 17 '20 at 23:16
  • I fat fingered the regex, but corrected it. Look at latest version of the comment. – tdelaney Feb 17 '20 at 23:16
  • price = re.findall(r"-?\d+", text) that kind of works lol, I just get a bunch of empty arrays, no errors though. just printing this " [] " – oz.vegas Feb 17 '20 at 23:17
  • Show us `repr(text)` and then we'll have what we need. – tdelaney Feb 17 '20 at 23:18
  • price = re.findall("\d+","Foo -111 Bar 55", text) gives me this 'TypeError: unsupported operand type(s) for &: 'str' and 'int'' – oz.vegas Feb 17 '20 at 23:19
  • where do I put ```repr(text) ``` in my code? sorry fairly new to this – oz.vegas Feb 17 '20 at 23:20
  • Remove text from that. The third parameter to findall is a integer flags. – tdelaney Feb 17 '20 at 23:21
  • Right before doing `re.findall` you could do `print(repr(text))`. Repr escapes line feed and things so we get the exact string. – tdelaney Feb 17 '20 at 23:21
  • ``` screen1 = np.array(ImageGrab.grab(bbox=(x))) new_screen1 = process_img(screen1) cv2.imshow('window1', new_screen1) text = pytesseract.image_to_string(new_screen1) repr(text) ``` doesn't do anything? – oz.vegas Feb 17 '20 at 23:22
  • printing it returns empty quotes - '' – oz.vegas Feb 17 '20 at 23:24
  • if the number is 0 it returns this 'Total profit = 0' which is the part of the screen i am looking at – oz.vegas Feb 17 '20 at 23:26
  • I just added a picture of what the code is returning – oz.vegas Feb 17 '20 at 23:31
  • I posted an answer assuming the problem is in handling edge conditions. – tdelaney Feb 17 '20 at 23:32

1 Answers1

0

I don't think the problem is with the regex itself, except that it can be expanded to catch the negative sign. I've reworked your code to deal with error cases:

        screen1 = np.array(ImageGrab.grab(bbox=(x)))
        new_screen1 = process_img(screen1)
        cv2.imshow('window1', new_screen1)   
        text = pytesseract.image_to_string(new_screen1)

        if text:
            # test code to show the text to scan     
            print("text to scan:", repr(text))
            try:
                price = re.findall("-?\d+", text)[0]
                print(price)
                if cv2.waitKey(25) & 0xFF == ord('q'):
                    cv2.destroyAllWindows()
                    break
                else:
                    pass
            except IndexError:
                print("No price found")
        else:
            print("No text to scan")
tdelaney
  • 73,364
  • 6
  • 83
  • 116
  • gives me this error ' RecursionError: maximum recursion depth exceeded in comparison' – oz.vegas Feb 17 '20 at 23:37
  • just updated the entirety of my code, maybe that will help more – oz.vegas Feb 17 '20 at 23:40
  • In the regex? At least the text was printed. Was it empty? – tdelaney Feb 17 '20 at 23:40
  • , i just restarted my kernel. i get multiple long messages with that code – oz.vegas Feb 17 '20 at 23:44
  • text to scan: 'mainimg.py py y\n\nChart Trader\nEST 500 So &\n\n500\nTP: 400 SL: 200\nProtect Pro 30 |100\nTrailing Ste 100/200\n\nLongs Close All Shorts\n\nprocessed_img = cv2.cvtColor(or\nest ate et\n\neee sto\nprocessed_imgl = cv2.cvtColor(o\nprocessed_img = cv2.Canny(proc\npeste tee ets\n\n \n\nrr\nry\n\n \n\ndelete All Pending Orders ff\n\nLong Pos. No./L/1 | 0.01\nAv. price: 1.68500\nLong profit: 14.36\n\nShort Po – oz.vegas Feb 17 '20 at 23:46
  • I got `['500', '500', '400', '200', '30', '100', '100', '200', '2', '2', '2', '1', '0', '01', '1', '68500', '14', '36']` for that text. As for the recursion error, does python give the line with the problem? Is it the regex? This code fragment isn't recursive, so it may be in the calling code. – tdelaney Feb 17 '20 at 23:49
  • i have no idea, i don't get an error code, i forgot to pass the new coordinates that's why i was getting the massive block of whatever that was, it was reading the whole screen. with the code reading that small snippet i posted a picture of it runs no errors but i doesn't show anything at all, just that the code is running – oz.vegas Feb 17 '20 at 23:55
  • with the code i have it works and returns a float multiple times a second, as long as its above zero, if it goes below i get this 'IndexError: list index out of range' frustrating haha – oz.vegas Feb 17 '20 at 23:58
  • I missed the `print("No text to scan")` part when I first pasted my code but have updated it since. If you get nothing printed, it could be text was empty. – tdelaney Feb 17 '20 at 23:59
  • works now no error, but when the number is negative it wont return a number just a string, ''No text to scan'' when the number is negative. step in the right direction forsure – oz.vegas Feb 18 '20 at 00:07
  • i big time appreciate the help, do you know how i can return a negative value instead of 'no text to scan' – oz.vegas Feb 18 '20 at 00:40
  • The problem is that text is empty. It has nothing to do with the regex which can be independently tested with negative numbers. So, look further up the code. I'd move `ImageGrab.grab(bbox=(x)` onto a separate line, fire up my debugger (or litter the code with prints) and see what happens at every step. – tdelaney Feb 18 '20 at 19:51