1

I'm tying to make a multi agent implementation of the 3x3 grid (grid0) for traffic light control In the get_state function, I want to differ in the information that I sent to the RL agent in this function. So agent 1 get only the information of vehicles driving on edges heading towards intersection 1.

In my understanding the `get_state function is called for each agent.

How can I distinguish between agents? Is it possible to do something like this?

agent_id = get_agent_id()
    if agent_id =0
        #return 'all info of vehicles on edges heading to traffic light1
    if agent_id =1
        ...

is there any way or function like this (agent list or something) to distinguish between the different agents in the get_state function?

And second, are the agent_id's the same as the traffic light id's (intersection_id's)? (And how do I assign a different agent to each intersection? Now I just use the default grid0 scenario, but I like to use multi-agent environment).

Thanks in advance!

Ashkan
  • 238
  • 1
  • 10
TNelen
  • 55
  • 1
  • 7

1 Answers1

1

1- In Flow, to handle multi-agent situations, in some methods (e.g. in get_state()) instead of returning the state information of a single agent as an np.array, we return a dictionary of states (with agent_id as key and agent_state as the value of the dictionary).

So you can do something like this:

    def get_state(self):

        agent_state_dict = {}
        i = 0
        for intersection, edges in self.scenario.get_node_mapping():
            i = i + 1
            agent_id = self.agent_name_prefix + str(i) # self.agent_name_prefix is defined as string "intersection"

            speeds = []
            dist_to_intersec = []
            traffic_light_states = []

            ..... code .....

            # construct the state (observation) for each agent
            observation = np.array(
                np.concatenate([
                    speeds, dist_to_intersec, traffic_light_states  

            # each intersection is an agent, so we will make a dictionary that maps form "self.agent_name_prefix+'i'" to the state of that agent.
            agent_state_dict.update({agent_id: observation})

        return agent_state_dict

The agent_state_dict is the dictionary that maps from agent_id to `observation' (which is the state)

2- Now to answer your second question, to define intersections as agents (so you will have multi-agent scenario), all you need to do is to define the corresponding RLlib functions (get_state, action_space, observation_space, compute_reward, and _apply_rl_actions) for an intersection. If you do this, you will have a complete env for multi-agent.

Ashkan
  • 238
  • 1
  • 10
  • Hi, thank you for your answer. I get error print that my enivronment object has no attribute 'agent_name_prefix'. Is there anything that i need to add in my enivonment to make it work? – TNelen Jul 03 '19 at 09:24
  • And how do i assign a different agent to each intersection? Now i just use the same way as the default grid0 scenario, but i use multi agent environment. – TNelen Jul 03 '19 at 11:11
  • @TNelen `agent_name_prefix` is an attribute that I have defined in my code. You don't need it actually. It's the string `'intersection'`, so that when it is concatenated with `i`, we will get something like `'intersection15'` – Ashkan Jul 04 '19 at 18:39
  • I edited my answer to also answer your second question in the comment. – Ashkan Jul 04 '19 at 18:57