1

I'm trying to use reinforcement learning in Webots with keras-rl, but resetting the world with supervisor bot kills the whole python process. Is there a way to reset the simulation without killing the process?

I'm running Ubuntu and using IntelliJ Idea for running the bot.

robot.simulationReset() # this resets the simulation
# here the whole process dies

edit. The basic structure I'm having now is like this

Also tried to start and kill the whole simulator with subprocess.Popen, but it didn't work because the controller does not reconnect to simulator. It also cannot be initialized again because of some internal logic preventing initialization more than once.

Master
  • 11
  • 2

1 Answers1

1

Once controllers are connected with Webots, they will be killed for sure when the simulation quits/reloads/reverts. This is a strong rule in Webots.

You should think on your design differently. Why not having an "uber-script" containing the reinforcement learning stuff and running Webots and its controllers as slaves (setup Webots simulations, send data to Webots controllers, get results, create Webots instances or communicate with existing ones, etc.)?

FabienRohrer
  • 1,794
  • 2
  • 15
  • 26
  • That's pretty much what I'm asking: how to run something on a higher level so the bot controller wouldn't kill the runner? The current implementation is a training script that imports the robot module, which makes the connection to webots simulator. I wouldn't want to invent the robot communication again between the "uber-script" and the robot "slave". But I also wouldn't like to restart the whole simulator for resetting it. – Master Sep 04 '19 at 16:54
  • In this case, what not creating a single Supervisor; at this end of an iteration it can simply reset the robot motors, translate it at its initial position and reset its physics (cf. the Supervisor API). In such case, no need to call the simulation quits/reloads/reverts functions, so the controller will never be reset. – FabienRohrer Sep 09 '19 at 06:29
  • I agree with Master, a full simulation reset without deconnnecting exern controllers is essential for advanced RL environement since often time not only the robot is spawned, but also external (random) objects. Maybe we can have an optional argument in the reset call : preserveControllers ? – Aerodynamic Sep 15 '20 at 15:10