-1

I want to be able to get an elements position on the screen of my computer then click it using pyautogui. However the only current way to do this reliably is to use puppeteer(pyppeteer) and screenshot the element then click it with pyautogui's png system.

Pyautogui's position system is always slightly off if I put in the coordinates that getBoundingClientRect() spits out for the element, I suspect it's to do with offsets for my taskbar etc but I want to be able to consistently convert between the two coordinate systems no matter the sizing of the webpage (whether it's a small box or full screen).

Is there python methods /ways of doing this?

For an example of the javascript code: document.querySelector("<class name>").getBoundingClientRect() which can be run in inspect element on chromium browsers etc.

Jimmy J.K.
  • 25
  • 6

1 Answers1

0

I found a decent way of doing. Namely get a chosen test elements coordinates using .getBoundingClientRect() then ask pyppeteer to take a screenshot of those coordinates. Then ask pyautogui to locate that png image and compare the pyppeteer coordinates to the pyautogui stated coordinates, just save the differences as a global variable etc (these in case it wasn't clear are the offsets of your current browser configuration) and you have a system for converting browser coordinates into pyautogui coordinates

Jimmy J.K.
  • 25
  • 6