0

I am trying to get inputs when I am playing the game (Warcraft III to be exact). Due to it being an RTS game it requires a high number of inputs in short periods. I am trying to use pyWinhook to get the inputs, mss to get screenshots and cv for image processing.

I have tried disabling image processing just after getting inputs to decrease the amount of work for the script. Nonetheless, it doesn't make much difference. I have also tried to decrease graphic details in-game - no difference also.

I am suspecting that thread is throttling due to high amount of inputs, and not very good hardware on which it's used (I am working at Lenovo X240 with 4GB of RAM).

import os

import cv2
import mss
import numpy as np
import pyWinhook as pyHook
import pythoncom


def main():
    # file names for training data arrays
    file_name = 'training_data.npy'
    copy_file_name = 'training_data_copy.npy'

    # deciding if previous file with data is saved. If yes, it is opened. If not it's created
    if os.path.isfile(file_name):
        print('File exists, loading previous data!')
        print(os.path.realpath(file_name))
        training_data = list(np.load(file_name, allow_pickle=True))
        np.save(copy_file_name, training_data)
    else:
        print('File does not exist, starting fresh!')
        training_data = []

    # getting screenshot and processing it for optimal machine learning processing
    def get_screen():
        with mss.mss() as sct:
            screen = np.array(sct.grab((0, 0, 1366, 768)))
        screen = cv2.cvtColor(screen, cv2.COLOR_BGR2GRAY)
        screen = cv2.resize(screen, (136, 76))
        return screen

    # saving data after acquiring 2500 sets of inputs and screenshots
    def save_data(screen, output):
        training_data.append([screen, output])
        if len(training_data) % 2500 == 0:
            print(len(training_data))
            np.save(file_name, training_data)
        print("Frames taken: " + str(len(training_data)))

    # getting inputs and screen on mouse event
    def OnMouseEvent(event):
        screen = get_screen()
        action = [event.Position, event.Wheel]
        output = action
        save_data(screen, output)
        return True

    # getting inputs and screen on keyboard event
    def OnKeyboardEvent(event):
        screen = get_screen()
        output = event.Key
        save_data(screen, output)
        return True

    # create a hook manager
    hm = pyHook.HookManager()

    # watch for all mouse events
    hm.MouseAll = OnMouseEvent
    hm.KeyUp = OnKeyboardEvent

    # set the hook
    hm.HookMouse()
    hm.HookKeyboard()

    # wait forever
    try:
        pythoncom.PumpMessages()
    except KeyboardInterrupt:
        pass

    # looping getting data
    while True:
        pass


if __name__ == '__main__':
    main()


I want to optimize getting inputs as much as possible because right now inputs in-game are lagging and hanging (which is not proper, because optimal inputs are the basics for machine learning modules).

I am fully aware that it can be an unsolvable problem due to slow hardware and too high amount of inputs per minute, but I want to try anyway.

  • You could have a look at [this GTA V](https://pythonprogramming.net/game-frames-open-cv-python-plays-gta-v/) tutorial by sentdex. I think he had the same problem – Uli Sotschok Aug 27 '19 at 13:03
  • I was basing on this tutorial :) If I remember correctly, there was a problem with getting inputs without focus on terminal. I am getting those, but without needed smoothness. Also, inputs in my case are far more complicated. – Damian Ziółkowski Aug 27 '19 at 13:05

0 Answers0