Locate color position on screen?

Question

I want to click on a specific color on the screen with pyautogui, but for that I need its position, and I can't find any useful information about the topic. I'm trying to make a Piano Tiles autoclicker and for that I've thought about identifying the tiles' color and clicking it.

Answer 1

You can find color position with pyautogui:

import pyautogui

color = (255, 255, 255)

s = pyautogui.screenshot()
for x in range(s.width):
    for y in range(s.height):
        if s.getpixel((x, y)) == color:
            pyautogui.click(x, y)  # do something here

Answer 2

Consider making a screenshot of smaller area to identify pixels faster.

pyautogui.screenshot(region=(0,0, 300, 400))

The argument is a four-integer tuple of the left, top, width, and height of the region to capture. You can even grab only one pixel of each tile to make it work better. I don't think making a screenshot of the whole screen would be a great idea, especially when tiles goes fast.

How I would do it:

1. use pyautogui.position() to get coords of one pixel of each region where tiles appears (assuming color of tile is solid and is not changing during the game)
2. use getpixel() to obtain the RGB values of tile pixel
3. check in loop if pixels with coordinates from step 1 have the same RGB values you obtained in step 2.
4. Call pyautogui.click() if yes

Answer 3

This is my first post! :) I had the same problem, but I found a solution. My code is probably not following any programming standard, but it is working hahaha! I started programming in Python 2 months ago (some experience 20 years ago (QBasic/C/Java), but never anything professional). Please tell me if is working for you and if there is anything that I can improve. I hope I can help somebody with this post, since this site has been helping me so much in the last 2 months!

def checkForRGBValues(start=(0,0), end=(50,50), R=255, G=255, B=255):

    x = int(end[0]-start[0] + 1)
    y = int(end[1]-start[1] + 1)
    # print(x)
    # print(y)
    for x in range(start[0], end[0] + 1, 1):
        for y in range(start[1], end[1] + 1, 1):
            print(str(x) + "/" + str(y))

            if pyautogui.pixel(x, y)[0] == R and pyautogui.pixel(x, y)[1] == G and pyautogui.pixel(x, y)[2] == B:
                print("Color found")
                with open("color_found.txt", "a") as file_found:
                    file_found.write(str(x) + "/" + str(y))
                    file_found.write("\n")
            else:

                with open("color_not_found.txt", "a") as file:
                    file.write(str(x) + "/" + str(y))
                    file.write("\n")

            y = y + 1
        x = x + 1



checkForRGBValues((150,200), (200,250), R=255, G=0, B=0) #if nothing (0,0), (50,50)

Answer 4

Here is another version that counts how many pixels are in the region:

import pyautogui

def checkForRGBValues(start=(0,0), end=(50,50), R=255, G=255, B=255): #start/end/r/g/b value, I put some standard values for testing

    x = int(end[0]-start[0] + 1) #calculates x value between start and end
    y = int(end[1]-start[1] + 1)  #calculates y value between start and end
    how_many_pixels_found = 0 #nothing found yet 
    for x in range(start[0], end[0] + 1, 1): #loops through x value
        for y in range(start[1], end[1] + 1, 1): #loops through y value
            if pyautogui.pixel(x, y)[0] == R and pyautogui.pixel(x, y)[1] == G and pyautogui.pixel(x, y)[2] == B: #checks if the wanted RGB value is in the region
                how_many_pixels_found = how_many_pixels_found + 1 #adds one to the result

            y = y + 1
        x = x + 1
    return how_many_pixels_found #returns the total value of pixels with the wanted color in the region.

x = checkForRGBValues((150,200), (200,250), R=60, G=63, B=65)
print(x)