1

Thanks to read this. I have developed chess AI like AlphaGo Lee or AlphaGo Zero. I have used Python and tensorflow. The chess AI is consist of Montecarlo-Tree-Search, policy network and value network.

I did learning for policy and value network for Montecarlo tree search. There is no problem. but, Each Simulation in Montecarlo tree search is too slow. So I want to boost each simulation speed.

I have known that python doesn't share object because of GIL. I really need to help for that. If you guys have a experience about share object in python multiprocessing, Please share your experience.

I post summary code below this page.

p.s : I'm not good at english. So, If you are uncomfortable at reading this page, that's my fault. please understand that. 

class monte
#I want to share Tree in multiprocessing
tree = Tree()

def doMontecarloTreeSearch:
    while numberOfsimulation:
        #I want to boost speed each simulation
        # but each search() computing neural network to make new node
        # so they spend much time.
        search()

def search:
    #node is created in each selection and is added in tree
    while is_gameover():
        selection()
        evaluation()
    backpropagation()

def selection
    #add best value node in Tree
def evaluation
    #each node is evaluated for expasion
def backpropagation
    # after gameove, leaf node backpropagate gameresult
    # and patent nodes are updated util parent node is root node
Dennis Soemers
  • 8,090
  • 2
  • 32
  • 55
Jean Michelle
  • 11
  • 1

1 Answers1

0

I have tried something similar for a Go engine and i am sorry for trying to do so with a Python project.

i started with speeding things up by using Cython and ended up with 90% Cython style c code and 10% Python code. Speedup in orders of 70x to 230x depending on the benchmarked part, average gameplay speedup was around 110x.

after that i started to work with multitasking, unfortunately certain parts of Python are, to be nice, rather uncooperative and somewhat slow. Still there are ways to get a net profit, depending on what you want to do.

Generating selfplay games
works rather nicely, i created a setup where i had:

  • 1 worker handling general flow, deciding amount of game generator worker thread when to train new model etc etc
  • 2 workers feeding gpu with positions to be evaluated
  • x workers playing games, either to generate self-play data or to evaluate model strength
  • 1 worker storing all the selfplay game to be used for training

  • 1 worker training a new model

    it was able to keep gpu around 80% occupied

multi threaded player
i tried several approaches with workers while minimizing data exchange between them because that is a huge bottleneck. in general it comes down to a master controller with a gametree dictating other workers to explore a certain node. having a compact board representation to send data between workers is of the essence!

the best engine i made had one master gametree controller and x slave tree controllers where the master keeps track of all the nodes visited and the slave requests if it may explore a certain node. it got kind of complicated but that way i was able to prevent any duplicate node visits. each slave had several workers that did the actual exploring of a node and that way i got decent gpu utilization.

Future
I i had to do it again and Python is a requirement i would create nice interaction for it and integrate some other language. for example you can integrate c/c++ with cython

MaMiFreak
  • 789
  • 2
  • 11
  • 26