Questions tagged [monte-carlo-tree-search]

Monte-Carlo Tree Search is a best-first, rollout-based tree search algorithm. It gradually improves its evaluations of nodes in the trees using (semi-)random rollouts through those nodes, focusing a larger proportion of rollouts on the parts of the tree that are the most promising. This tag should be used for questions about implementation of this algorithm.

Monte-Carlo Tree Search (MCTS) is a best-first tree search algorithm. "Best-first" means that it focuses the majority of the search effort on the most promising parts of a search tree. This is a popular algorithm for automated game-playing in Artificial Intelligence (AI), but also has applications outside of AI.

For an introduction into the types of search trees that MCTS is typically used for, see minimax.

Older algorithms, such as minimax and alpha-beta-pruning, evaluate a node by systematically searching all the nodes below it, which can be very computationally expensive. The basic idea of MCTS is to rapidly approximate such systematic evaluations of nodes by using (semi-)random rollouts and averaging the evaluations at the end of those rollouts. Over time, the algorithm will focus a larger amount of search effort on parts of the search tree that appear promising based on the earlier rollouts. This enables the algorithm to gradually improve the approximations of the evaluations in those parts of the tree. Additionally, the algorithm gradually grows a tree by storing a number (typically 1) of nodes in memory for every rollout. The parts of the tree that are stored in memory are not traversed using a (semi-)random strategy, but using a different ("Selection") strategy. This guarantees that, given an infinite amount of computation time, the algorithm converges to optimal solutions.

An extensive survey, describing many enhancements and applications of the algorithm, can be found in:

  • Browne, C., Powley, E., Whitehouse, D., Lucas, S., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., and Colton, S. (2012). A Survey of Monte Carlo Tree Search Methods. Computation Intelligence and AI in Games, IEEE Transactions on, Vol. 4, No. 1, pp. 1-43.

The algorithm was originally proposed in:

  • Kocsis, L. and Szepesvári, C. (2006). Bandit Based Monte-Carlo Planning. Proceedings of the 17th European Conference on Machine Learning (ECML 2006) (eds. J. Fürnkranz, T. Scheffer, and M. Spiliopoulou), Vol. 4212 of Lecture Notes in Computer Science, pp. 282-293, Springer Berlin Heidelberg.
  • Coulom, R. (2007). Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. Computers and Games (eds. H. J. van den Herik, P. Ciancarini, and H. H. L. M. Donkers), Vol. 4630 of Lecture Notes in Computer Science, pp. 72-83, Springer Berlin Heidelberg.
79 questions
0
votes
1 answer

How to fix my MCTS-based AI for the Blokus game?

I'm working on an AI for the Blokus game using the Monte Carlo Tree Search (MCTS) algorithm. I've implemented the core of the algorithm, including the selection, expansion, simulation, and backpropagation phases. However, I'm facing some performance…
0
votes
0 answers

Explaining the red elements with BS that appear when training my Keras model

I am pretty new to keras, but have done the Mnist-dataset and achieved 99.2% accuracy, so i at least know how to train a model. The problem i have with this is that it looks quite different from the other training samples i have done. I pulled a git…
0
votes
0 answers

Monte Carlo Tree Search to find the highest value leaf node

I'm currently trying to code the Question 2 here: Assignment All I need is a rough outline of how to go about this. All the examples I have seen of MCTS seem to be about 2X2 board games and I don't understand how to extrapolate that to this example.…
0
votes
2 answers

What is most efficient way to access nodes of a tree stored in a NumPy array

Imagine we have a tree of values stored in a NumPy array. For example - In [1]: import numpy as np In [2]: tree = np.array([[0, 6], [0, 4], [1, 3], [2, 9], [3, 1], [2, 7]]); In [3]: tree.shape Out[3]: (6, 2) Each node in the tree is row in the…
user2309803
  • 541
  • 5
  • 15
0
votes
0 answers

Modifying the Monte Carlo Tree Search method for the game Entropy

I am thinking about using this method (Monte Carlo Tree Search) for this game Entropy. But I am stuck at the point that MCTS will let you create a child node from a move that you can play. But if you play as Chaos, you will have to put the piece…
0
votes
0 answers

Why does this MCTS code miss forced wins?

I'm trying to make a simple Monte Carlo Tree Search program (python3) that can take a given board and the player whose turn it is and will return the optimal move for them. I've written one that works somewhat, but while testing it with Tic-Tac-Toe…
0
votes
0 answers

how did alphago visualize markov decision process?

Does anyone know how to visualize the markov decision process and monte carlo tree search like in this video? https://www.youtube.com/watch?v=MgowR4pq3e8&ab_channel=ArxivInsights At 7:12 in the above video, each step span is visualized with a new…
0
votes
0 answers

How to include Temporal Difference in Monte Carlo Tree Search?

Im wondering how TD can be included in MCTS to enhance its learning? Most TD applications use the Reward obtained in the next state S', however, in MCTS rewards are obtained after a whole rollout, so, how can TD be implemented? Would it be something…
0
votes
1 answer

Mone Carlo Tree Search and terminal Nodes handling

I'm trying to implement AlphaZero on a new game using this repository. I'm not sure if they are handling the MCTS search tree correctly. The logic of their MCTS implementation is as follows: Get a "canonical form" of the current game state.…
0
votes
1 answer

Optimizing fighting bots

Imagine you are supposed to write an algorithm for a bot that will fight other similarly prepared bots. Your bot has 200 HP for a whole fight and gets a set value of 12 energy points per every round (maximum 100 rounds). Your bot have to attack…
0
votes
0 answers

Trying to save a binary tree as vector occurs memory issue (C++)

I'm trying to implement MCTS using C++, and in order to save (in a file) all the moves played by the simulations, I implemented a "arbrecontigu" class to construct a vector of element (which contain informations used in our MCTS : 'Brix' is for…
0
votes
1 answer

Issues with unit testing

I want to test a function, but I am definitely struggling at this one. The function loops through boards last row if NO_PLayer is valid location. def validLocations(board): validLocationsArr = [] column = 0 row = 0 while column <…
0
votes
1 answer

MCTS Agent making bad decisions on Tic-Tac-Toe

I've been working on a MCTS AI for a couple days now. I tried to implement it on Tic-Tac-Toe, the least complex game I could think of, but for some reason, my AI keeps making bad decisions. I've tried change the values of UCB1's exploration…
0
votes
0 answers

"ValueError: Empty module name" when using pathos.multiprocessing

As a preface, I'm utilizing Monte Carlo Tree Search to run a model-based reinforcement learning task. Basically I have an agent foraging in a discrete environment, where the agent can see out some number of spaces around it (I'm assuming perfect…
0
votes
1 answer

Ways to speed up Monte Carlo Tree Search in a model-based RL task

This area is still very new to me, so forgive me if I am asking dumb questions. I'm utilizing MCTS to run a model-based reinforcement learning task. Basically I have an agent foraging in a discrete environment, where the agent can see out some…
DECK
  • 1
  • 1