1

For my Msc thesis I want to apply multi-agent RL to a bus control problem. The idea is that the busses operate on a given line, but without a timetable. The busses should have bus stops where passengers accumulate over time and pick them up, the longer the interval between busses, the more passengers will be waiting at the stop (on average, it's a stochastic process). I also want to implement some intersections where busses will have to wait for a green light.

I'm not sure yet what my reward function will look like, but it will be something along the lines of keeping the intervals between busses as regular as possible or minimising total travel time of the passengers.

The agents in the problem will be the busses, but also the traffic lights. The traffic lights can choose when to show green light for which road: apart from the busses they will have other demand as well that has to be processed. The busses can choose to speed up, slow down, to wait longer at a stop or to continue on normal speed.

To be able to put this problem in a RL framework I will need an enviroment and suitable RL algorithms. Ideally I would have a flexible simulation environment to re-create my case study bus line and connect this to of-the-shelf RL algorithms. However, so far I haven't found this. This means I may have to connect a simulation environment to something like an OpenAI gym myself.

Does anyone have advice for which simulation environment may be suitable? And if it's possible to connect this to of-the-shelf RL algorithms?

I feel most comfortable with programming in Python, but other languages are an option as well (but this would mean considerable extra effort from my side).

So far I have found the following simulation environments that may be suitable:

NetLogo
SimPy
Mesa
MATSim (https://www.matsim.org)
Matlab
CityFlow (https://cityflow-project.github.io/#about)
Flatland (https://www.aicrowd.com/challenges/neurips-2020-flatland-challenge/)

For the RL algorithms the options seem to be:

  1. Code them myself
  2. Create the environment according to the OpenAI gym API guidelines and use the OpenAI baselines algorithms.

I would love to hear some suggestions and advice on which environments may be most suitable for my problem!

cello
  • 5,356
  • 3
  • 23
  • 28
CEJoe
  • 11
  • 1
  • If you are already comfortable in Python, there is a specific framework you could look at - GAMA at https://gama-platform.github.io/. I see Mesa on your list, which is another python project. You may also be interested in RePast (which has a python version) – JenB Sep 23 '20 at 12:59
  • Thanks @JenB, the GAMA project seems interesting! – CEJoe Sep 25 '20 at 09:34

1 Answers1

0

You can also check SUMO as a traffic simulator and RLLib library for multi-agent reinforcement learning.

funie200
  • 3,688
  • 5
  • 21
  • 34
Arpad
  • 1
  • 1
  • Thanks, the RLlib looks promising! Also, by searching for it I found another option as well: the Tensorforce library, which seems to be easy to implement and has (non-native) multi-agent support. – CEJoe Oct 09 '20 at 12:04