Integration with open-ai gym / stable baselines

Question

Open AI gym is best posed to become the benchmarking standard for ML, as well as the stable baselines algorithms are one of the most popular ML libraries. Is there any guide on how to integrate Webots with the gym environment for that stable-baselines can be used ? The gym env exposes some fucntions like step and reset, however I'm not sure how these could be used to control webots. If someone with more webots API/external control knowledge could lay out a path for integration, I would much appreciate it - even write a guide on how to integrate it for others (if not already done) ?

Answer 1

I don't have a clear step by step guideline but here are a few facts that can help you to integrate Webots with open-ai gym:

1. Webots does provide a Python API: https://cyberbotics.com/doc/guide/using-python
2. The recent versions of Webots provide the possibility to use what is called 'extern controller', this feature let you integrate a controller in any other application and let the application start the controller by itself instead of Webots starting it: https://www.cyberbotics.com/doc/guide/running-extern-robot-controllers
3. Some users have already tried to integrate Webots with open-ai gym: https://github.com/talregev/gym-webots

Finally, if you plan to make an example and agree to make this sample public (i.e. on a Github account) let me know I might be able to follow the project and help you with your Webots specific questions (I am interested in the subject too but don't have enough time to invest in this to do it).

Answer 2

Now, you can check out a simple Webots example of an OpenAI Gym interface integration:
https://www.cyberbotics.com/doc/guide/samples-howto?version=master#openai_gym-wbt

Also, check out the Deepbots framework:
https://github.com/aidudezzz/deepbots