0

When I am learning tutorial 9, I am confusing the rl_actions. Because on the program, the rl_actions is not initialized and definition. Why there is a 'rl_actions' parameter of the _apply_rl_actions function and compute_reward function? I also check the vehicle kernel code, about apply_acceleration function. The original one is:

def apply_acceleration(self, veh_ids, acc):
        """See parent class."""
        # to hand the case of a single vehicle
        if type(veh_ids) == str:
            veh_ids = [veh_ids]
            acc = [acc]

        for i, vid in enumerate(veh_ids):
            if acc[i] is not None and vid in self.get_ids():
                this_vel = self.get_speed(vid)
                next_vel = max([this_vel + acc[i] * self.sim_step, 0])
                self.kernel_api.vehicle.slowDown(vid, next_vel, 1e-3)
user11795736
  • 69
  • 5

1 Answers1

1

Look into flow/envs/base_env.py in the step method, this is where apply_rl_actions and compute_reward are called. All these 3 methods take as parameter the actions rl_actions to apply to the agents. These actions are provided by the RL algorithm. The shape of rl_actions is the one provided in the action_space method of your environment.

The RL algorithm automatically calls your step method at each step, giving it the actions to be applied. Flow's environments are actually encapsulated within a Gym environment, which is given to the RL algorithm. The RL algorithm can work with any Gym environment, which allows it to be very general, since all Gym environments have methods such as step, reset etc. If you want to know more about how this work, look into how to train a custom Gym environment.

math.husky
  • 193
  • 1
  • 8