3

If I want to investigate an RL algorithm for robotics, how should I be using Gazebo and OpenAI Gym to test, train and benchmark the algorithm? Should I start with OpenAI Gym and take algorithms with good scores into the Gazebo environment for real-world scenarios?

shivangg
  • 521
  • 8
  • 15
  • Why not both? ;-) [Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo](https://arxiv.org/pdf/1608.05742.pdf) – Toke Faurby Apr 21 '17 at 06:50
  • @TokeFaurby This pdf is what that made me ask this question. If both may be used in combination, should the RL algo be first tested on OpenAI Gym and the ones with high score be tried on Gazebo? Or is there a better way?[This was also my initial question but the suggested edit made it explore other possibilities] – shivangg Apr 27 '17 at 08:03
  • welcome! I am working on a similar project. I would start implementing the [Continuous control with deep reinforcement learning](https://arxiv.org/abs/1509.02971) DeepMind paper (using OpenAI gym). Once you have this working I would move on to Gazebo. PS please add all relevant information to the OP (i.e. you should have mentioned the paper, it would have made helping easier) – Toke Faurby May 01 '17 at 02:48

1 Answers1

5

Factors to consider when picking the framework to work within

  • How much time will it require to get up to speed with whatever you choose?
  • Will performance in the environment reliably predict performance on an actual robot?
  • How much will it cost?

If you want to have your approach eventually work on an actual robot, you should be testing in an environment that closely simulates your target environment and platform. OpenAI gym's first party robot simulation environments use MuJuCo, which is not free. Further, these simulations are more for toy control setups than actual robotics problems. You would be better served writing ROS nodes and simulating your problem in Gazebo. You might also look at something like Erle Robotics' gym-gazebo tool which will let you bridge between gym and ROS.

Nick Walker
  • 790
  • 6
  • 19