i am building an RL agent for which the model is defined:
def build_model(states, actions):
azioni = list(actions)
model = Sequential()
model.add(Dense(4, activation='relu', input_shape=[len(azioni)]))
model.add(Dense(4, activation='relu'))
return model
The action.space is:
self.action_space=gym.spaces.Tuple(tuple([gym.spaces.Discrete(3)]*4 ))
my action space consists of 4 action for 4 agent, where agent is a list of 4 object agent.
Then define Agent:
def build_agent(model, actions):
policy = BoltzmannQPolicy()
memory = SequentialMemory(limit=50000, window_length=1)
dqn = DQNAgent(model=model, memory=memory, policy=policy,
nb_actions=actions, nb_steps_warmup=10, target_model_update=1e-2)
return dqn
Whine I try to build the Agent:
dqn = build_agent(model, actions)
I get this error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-98-512746ffa1dc> in <module>
11 return dqn
12
---> 13 dqn = build_agent(model, actions)
14 dqn.compile(Adam(lr=1e-3), metrics=['mae'])
15 dqn.fit(env, nb_steps=50000, visualize=False, verbose=1)
<ipython-input-98-512746ffa1dc> in build_agent(model, actions)
8 memory = SequentialMemory(limit=50000, window_length=1)
9 dqn = DQNAgent(model=model, memory=memory, policy=policy,
---> 10 nb_actions=actions, nb_steps_warmup=10, target_model_update=1e-2)
11 return dqn
12
~/PycharmProjects/pythonProject6/venv/lib/python3.6/site-packages/rl/agents/dqn.py in __init__(self, model, policy, test_policy, enable_double_dqn, enable_dueling_network, dueling_type, *args, **kwargs)
109 raise ValueError('Model "{}" has more than one output. DQN expects a model that has a single output.'.format(model))
110 if tuple(model.output.shape) != list((None, self.nb_actions)):
--> 111 raise ValueError('Model output "{}" has invalid shape. DQN expects a model that has one dimension for each action, in this case {}.'.format(model.output, self.nb_actions))
112
113 # Parameters.
ValueError: Model output "Tensor("dense_16/Relu:0", shape=(?, 4), dtype=float32)" has invalid shape. DQN expects a model that has one dimension for each action, in this case (1, 2, 1, 0).
I cheched the value of model.output.shape and action:
print(model.output_shape)
#(None, 4)
print(actions)
#(1, 2, 1, 0)
I don t know how to adapt the action list to assing to my agents with the model.output_shape. Since my space_action is given by a list where space_action[0] is the action of agent[0], action[1] is the action of agent[1] etc.
thanks for the assist.
