I have a script that is used to login in to a company-made application and click the right buttons like "continue", or "ok", etc. to perform a certain process. However, I have had to use screenshots of these buttons to click in order to do this using pyautogui. Is there any package or way to automate this process without using images. Maybe it can detect the text of the button and click it. I do not have identifiers for the buttons available and no access to the code/info behind the application. Let me know if you have any ideas. Thanks!
Asked
Active
Viewed 417 times
1 Answers
0
I have a few questions that may be helpful:
- Does the layout of the buttons change? If it's always the same you can just program the correct locations and timing and not worry about reading the screen.
If you really have to read the screen, look into optical character recognition (ocr). - Is the application keyboard accessible? If so, using Tab and Enter to activate the buttons is simpler than controlling the mouse. Also, if it was made by superstars you can use find (ctrl-F) to search for the text on the buttons and go to them.
This answer is pretty vague, but I can only be as specific as the question asked.
Jelly Joe
- 38
- 1
- 7
-
No that was a great response. So unfortunately there are no keys you can use and ctrl-f would not work in the program either. Really great lol. I will look into the OCR however, that was a good tip. – eddyferreira7 Jun 23 '22 at 12:29