TRPO - RL: I need to get a 8DOF robot arm to move to a specified point. I need to implement the TRPO RL code using OpenAI gym with Gazebo environment?

Question

TRPO - RL: I need to get a 8DOF robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environement. But I am unsure of how to write the code for the reward functons and the algorithm for the joint space motion. Please help.

Answer 1

Reward

Gazebo should be able to tell you the position of the end-effector link from which we can calculate the progress made towards a specified point after each step (i.e. positive if moving towards the goal, negative if away, and 0 otherwise). This alone should encourage the end-effector towards the goal.

You may want to confirm that the system is able to learn with just this basic reward first before considering other criterions such as smoothness (avoid jerking motions), handedness (positioning the elbows on the left/right) etc. These are significantly harder to specify and will have to be hand-designed according to your needs, possibly based on the joint states and/or some other derivatives that are available in your environment.

Motion

This will largely depend on your stack. I am adding this part in just as a passing comment, but for instance, if you are using ROS as your middleware, then you can easily integrate Move-It to handle all the movement for you.