Stack Overflow
Stack Overflow

Questions tagged [tf-agent]

43 questions
1
vote
0 answers

How to get distributions of LinearThompsonagent in tf_Agents

I am working on contextual bandits in tf_Agents and using the linearUCB agent and leanr thompson sampling agent. I can get the actions, but not sure how to get the distributions (over actions) out of the agents for a given timestep. I know linearUCB…
tjt
  • 620
  • 2
  • 7
  • 17
1
vote
0 answers

How to set breakpoint in TensorFlow uniform replay buffer?

I'm using tensorflow2 and dealing with tf_agents.replay_buffers.TFUniformReplayBuffer. As the comments of source codes suggest, each block of the replay buffer is consisted of several episodes: The TFUniformReplayBuffer stores episodes in `B ==…
zzl
  • 11
  • 1
1
vote
0 answers

InvalidArgumentError: Must have updates.shape = indices.shape + params.shape[1:] or updates.shape = [] in TF-Agents

I have a problem with this simple code. I am looking for an experience replay code that is compatible with graph operations and TF Functions. import tensorflow as tf import numpy as np !pip install tf_agents import tf_agents …
Mohammad.J
  • 33
  • 4
1
vote
2 answers

TF-Agents getting malloc error while training

I am running on a problem with malloc error, when trying to use tf-agents library for training a DQN. Specs: M1 mac os 12 TF 2.6.2, tf-agents 0.10.0 Python 3.8 (same results for 3.9) Im using custom Environment, wrapped into TF env. Everything else…
JanK
  • 31
  • 3
1
vote
0 answers

How to load policy saved with triggers.PolicySavedModelTrigger in tf Agent

I ran the SAC training code as shown in this demo [1]. When training finishes, inside the policies folder, there are a few folders: checkpoints, collect_policy, greedy_policy, policy. I would like to load the policy from disk for evaluation. From…
Quan Vuong
  • 1,919
  • 3
  • 14
  • 24
0
votes
1 answer

Tensorflow DDPG agent. Shape mismatch in critic loss function using DDPG agent. Mismatch between td_targets and q_values tensors

I have an issue when computing the mean squared error in the critic loss function using a DDPG agent. The error message I receive indicates a shape mismatch between the expected tensor shape and the actual tensor shape in the critic loss function of…
Peter Renz
  • 23
  • 5
0
votes
1 answer

How can I improve the metrics of my DQN agent in tensorflow?

I'm working on a deep reinforcement learning project with TensorFlow and I am struggling with the training of a DQN agent of tf_agents module. My project aims to simulate a fiscal society where there are tree possible actions: pay taxes, pay more…
Willy
  • 1
  • 1
0
votes
0 answers

Categorical DQN Agent with masking in tf_agents raises a structure issue

I'm currently training a RL agent with tf_agents in order to play a card game. As it worked well with a DQN agent, I tried to improve my results by using a categorical DQN agent. This card game includes valid/invalid actions, so I provided a mask to…
Smaguy
  • 1
  • 1
0
votes
0 answers

How to reset the weights of TF-Agents DDQN Network?

Assume that you have successfully trained a DDQN agent for a number of iterations in TF-Agents. Now, you need to reset the neural network weights (of the Q-Network; apparently) to the form it was initialized to and continue the training from there…
Maad A.Galil
  • 1
  • 5
0
votes
0 answers

tf_agents reset environment using actor

I'm trying to understand how to use Actor class in tf_agents. I am using DDPG (actor-critic, although this doesn't really matter per say). I also am learning off of gym package, although again this isn't fully important to the question. I went into…
brian_ds
  • 317
  • 4
  • 12
0
votes
1 answer

how to use tf_agents.policies.ou_noise_policy to create an ou process object?

I tried to use from tf_agents.trajectories.ou_noise import OrnsteinUhlenbeckProcess to create ou process but this library doesn't exist anymore . after research, I found this tf_agents.policies.ou_noise_policy, wondering how to use this to create…
clockbird
  • 1
0
votes
0 answers

Incorporate batching directly into PyEnvironment in tf-agents

Instead of writing a PyEnvironment and batching it (using BatchedPyEnvironment), I would like to write a PyEnvironment that is directly in a batched format. This means my observation should be of shape (batch_size, ..), and my actions, discounts and…
Henrik
  • 1
0
votes
0 answers

How to pass sequence to train agent QRnnNetwork (Dimensions must be equal erro)

I have set-up a tf_agent with a normal neural network as a q-net to learn trajectories which works fine. However, I'd now like to try a QRnnNetwork and train/learn from sequences of events but can't get it to work. The action and observation spec in…
vincentp
  • 31
  • 6
0
votes
1 answer

Convert an TF Agents ActorDistributionNetwork into a Tensorflow lite model

I would like to convert the ActorDistributionModel from a trained PPOClipAgent into a Tensorflow Lite model for deployment. How should I accomplish this? I have tried following this tutorial (see section at bottom converting policy to TFLite), but…
Setjmp
  • 27,279
  • 27
  • 74
  • 92
0
votes
0 answers

Reinforcement Learning with tf-agents py_environment and tf_py_environment

I would like to use the tf-agents RL, but there's two environments: py_environment and tf_py_environment. The former uses numpy arrays for the states and actions while the latter uses tensors. My states have data type set -- clearly numpy can handle…
M. Fire
  • 117
  • 2
  • 6
1
2
3