Stack Overflow
Stack Overflow

Questions tagged [tensorflow-agents]

33 questions
8
votes
1 answer

ValueError: Could not find matching function to call loaded from the SavedModel

I am trying to load a tf-agents policy I saved via try: PolicySaver(collect_policy).save(model_dir + 'collect_policy') except TypeError: tf.saved_model.save(collect_policy, model_dir + 'collect_policy') Quick explanation for the try/except…
Taxel
  • 3,859
  • 1
  • 18
  • 40
7
votes
1 answer

Can tf.agent policy return probability vector for all actions?

I am trying to train a Reinforcement Learning agent using TF-Agent TF-Agent DQN Tutorial. In my application, I have 1 action containing 9 possible discrete values (labeled from 0 to 8). Below is the output from…
BING ZHAO
  • 96
  • 4
6
votes
1 answer

py_environment 'time_step' doesn't match 'time_step_spec'

I have created a custom pyenvironment via tf agents. However I can't validate the environment or take steps within it with py_policy.action I'm confused as to what is excepted from the time_step_specs I have tried converting to tf_py_environment via…
Boyang Zhang
  • 117
  • 7
4
votes
1 answer

Issue implementing q-rnn in tf-agents

I have been trying to build a rl agent using tf-agents in tensorflow. I experienced the issue in a custom built environment but reproduced it using an official tf colab example. The problem occurs whenever I try to use QRnnNetwork as the network…
devboydan
  • 51
  • 3
4
votes
1 answer

TF-agents - Replay buffer add trajectory to batch shape mismatch

I'm posting a question that was posted by another user and then deleted. I had the same question, and I found an answer. The original question: I am currently trying to implement a categorical DQN following this tutorial:…
David Braun
  • 782
  • 1
  • 9
  • 18
3
votes
1 answer

TFAgents: how to take into account invalid actions

I'm using TF-Agents library for reinforcement learning, and I would like to take into account that, for a given state, some actions are invalid. How can this be implemented? Should I define a "observation_and_action_constraint_splitter" function…
MarcoM
  • 1,093
  • 9
  • 25
2
votes
0 answers

Which policy to use after training RL agent

When running the Tensorflow agents notebook for the Soft Actor-Critic Minitaur, https://www.tensorflow.org/agents/tutorials/7_SAC_minitaur_tutorial, the following directories are created under /tmp: +tmp -eval -train +policies …
Daniel von Eschwege
  • 481
  • 1
  • 4
  • 10
2
votes
1 answer

tf_agents doesn't properly learn a simple environment

I successfully followed this official tensorflow tutorial for training an agent to solve the 'CartPole-v0' gym environment. I only diverged from the tutorial in that I did not use reverb, because it's not supported on Windows. I tried to modify the…
Gaberocksall
  • 359
  • 2
  • 13
2
votes
1 answer

How to give a tuple of actions to a TF-Agents environment within the step method?

I want to model my environment such that each action is made of 3 possible sub-actions. I've defined the _action_spec of my tf_agents.environments.py_environment.PyEnvironment as: self._action_spec = tf_agents.specs.BoundedArraySpec( …
Lostefra
  • 350
  • 4
  • 13
2
votes
1 answer

Tf-Agents ParallelPyEnvironment fails silently

I have written a custom environment so I can play around with reinforcement learning (PPO) and tf-agents. This works fine if I wrap my env ( which inherits from py_environment.PyEnvironment) in a TfPyEnvironment, but fails if I try to wrap it into a…
Taxel
  • 3,859
  • 1
  • 18
  • 40
1
vote
0 answers

tf_agents changing underlying suite_gym reward function

I'm trying to modify the MountainCarContinuous-v0 environment from suite_gym() because training is getting stuck in a local minima. The default reward function penalizes large actions which are preferred for optimal solving. So I would like to try…
brian_ds
  • 317
  • 4
  • 12
1
vote
1 answer

Error when saving model with tensorflow-agents

I am trying to save a model with tensorflow-agents. First I define the following: collect_policy = tf_agent.collect_policy saver = PolicySaver(collect_policy, batch_size=None) and then save the model like this: saver.save('my_directory/') This…
Enrique
  • 9,920
  • 7
  • 47
  • 59
1
vote
1 answer

PPOAgent + Cartpole = ValueError: actor_network output spec does not match action spec:

I'm trying to experiment with using tf_agents' PPOAgent in the CartPole-v1 environment, but I am recieving the following error upon declaring the agent itself: ValueError: actor_network output spec does not match action spec: TensorSpec(shape=(2,),…
Old_Frog
  • 11
  • 3
1
vote
1 answer

How to get probability vector for all actions in tf-agents?

I'm working on Multi-Armed-Bandit problem, using LinearUCBAgent and LinearThompsonSamplingAgent but they both return a single action for an observation. What I need is the probability for all the action which I can use for ranking.
Kushal Jain
  • 11
  • 2
1
vote
1 answer

Shape of _observation_spec and shape of _action_spec in the Tf-agents environments example

In the tensorflow documentation for TF-Agents Environments there is an example of an environment for a simple (blackjack inspired) card game. The init looks like the following: class CardGameEnv(py_environment.PyEnvironment): def __init__(self): …
sapo_cosmico
  • 6,274
  • 12
  • 45
  • 58
1
2 3