Monte-Carlo Tree Search is a best-first, rollout-based tree search algorithm. It gradually improves its evaluations of nodes in the trees using (semi-)random rollouts through those nodes, focusing a larger proportion of rollouts on the parts of the tree that are the most promising. This tag should be used for questions about implementation of this algorithm.
Monte-Carlo Tree Search (MCTS) is a best-first tree search algorithm. "Best-first" means that it focuses the majority of the search effort on the most promising parts of a search tree. This is a popular algorithm for automated game-playing in Artificial Intelligence (AI), but also has applications outside of AI.
For an introduction into the types of search trees that MCTS is typically used for, see minimax.
Older algorithms, such as minimax and alpha-beta-pruning, evaluate a node by systematically searching all the nodes below it, which can be very computationally expensive. The basic idea of MCTS is to rapidly approximate such systematic evaluations of nodes by using (semi-)random rollouts and averaging the evaluations at the end of those rollouts. Over time, the algorithm will focus a larger amount of search effort on parts of the search tree that appear promising based on the earlier rollouts. This enables the algorithm to gradually improve the approximations of the evaluations in those parts of the tree. Additionally, the algorithm gradually grows a tree by storing a number (typically 1) of nodes in memory for every rollout. The parts of the tree that are stored in memory are not traversed using a (semi-)random strategy, but using a different ("Selection") strategy. This guarantees that, given an infinite amount of computation time, the algorithm converges to optimal solutions.
An extensive survey, describing many enhancements and applications of the algorithm, can be found in:
- Browne, C., Powley, E., Whitehouse, D., Lucas, S., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., and Colton, S. (2012). A Survey of Monte Carlo Tree Search Methods. Computation Intelligence and AI in Games, IEEE Transactions on, Vol. 4, No. 1, pp. 1-43.
The algorithm was originally proposed in:
- Kocsis, L. and Szepesvári, C. (2006). Bandit Based Monte-Carlo Planning. Proceedings of the 17th European Conference on Machine Learning (ECML 2006) (eds. J. Fürnkranz, T. Scheffer, and M. Spiliopoulou), Vol. 4212 of Lecture Notes in Computer Science, pp. 282-293, Springer Berlin Heidelberg.
- Coulom, R. (2007). Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. Computers and Games (eds. H. J. van den Herik, P. Ciancarini, and H. H. L. M. Donkers), Vol. 4630 of Lecture Notes in Computer Science, pp. 72-83, Springer Berlin Heidelberg.
