Stack Overflow
Stack Overflow

Questions tagged [gradienttape]

147 questions
0
votes
1 answer

Compute Hessian of lossfunction in Tensorflow

I would like to compute the hessian of a loss function of a neural network in Tensorflow with respect to all the parameters (or trainable variables). By modifying the example code from the Tensorflow documentation…
user202542
  • 211
  • 1
  • 8
0
votes
0 answers

How to calculate second order gradient using GradientTape?

I am trying to calculate partial derivative for the below function, could you advise if this is the right way to calculate partial derivative using TensorFlow? f(x,y,z) = xy+z pow 2 + 1/x x = tf.Variable(2, name='x') y = tf.Variable(1, name='y') z…
bigskull
  • 739
  • 2
  • 13
  • 23
0
votes
1 answer

Tensorflow: Custom training loop using GPU slower than CPU

I am doing this custom training of a neural network in Colab, with and without GPU, and the training process is faster using the CPU, which makes me think that I am not parallelising the operations or missing something. I do not think it is because…
rrmaura
  • 1
0
votes
1 answer

Why is tf.GradientTape.jacobian giving None?

I'm using the IRIS dataset, and am following this official tutorial: Custom training: walkthrough In the Training loop, I am trying to gather the model outputs and weights in each epoch%50==0 in the lists m_outputs_mod50, gather_weights…
christospap
  • 13
  • 4
0
votes
0 answers

Tensorflow GradientTape computes only 'None' gradients with custom loss

So I have a model (A) that I´m training with a custom training procedure. It is supported by a second model (B). This means of course I have to use tf.GradientTape and compute + apply the gradients myself. However, it doesn´t work as expected and…
Marie M.
  • 170
  • 1
  • 3
  • 13
0
votes
3 answers

ValueError: tape is required when a Tensor loss is passed

As said in https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Optimizer?hl=en#minimize, the first parameter of minmize should satisfy the requirement, Tensor or callable. If a callable, loss should take no arguments and return the value…
Qiqin Zhan
  • 71
  • 1
  • 6
0
votes
1 answer

Internal error: Tried to take gradients (or similar) of a variable without handle data in Tensorflow

I am finetuning BERT for a binary sentiment analysis class using Tensorflow. I want to use a custom training loop/loss function. However, when I train the model I get the following error: ValueError: Internal error: Tried to take gradients (or…
Jane Sully
  • 3,137
  • 10
  • 48
  • 87
0
votes
2 answers

Custom GAN training loop using tf.GradientTape returns [None] as gradients for generator while it works for discriminator

I am trying to train a GAN. Somehow the gradient for the generator returns None even though it returns gradients for the discriminator. This leads to ValueError: No gradients provided for any variable: ['carrier_freq:0']. when the optimizer applies…
jost
  • 89
  • 5
0
votes
1 answer

gradient using tf.GradientTape() wrt inputs is None (Tensorflow 2.4)

This is my model. Im using Tensorflow 2.4.1. model = tf.keras.Sequential([ tf.keras.layers.Embedding(input_dim=1000, output_dim=64, name='embedding', …
Madhu Babu Adiki
  • 51
  • 3
0
votes
1 answer

Custom Training Loop for Tensorflow Variational Autoencoder: `tape.gradient(loss, decoder_model.trainable_weights)` Always Returns List Full of None's

I am trying to write a custom training loop for a variational autoencoder (VAE) that consists of two separate tf.keras.Model objects. The objective of this VAE is multi-class classification. As usual, the outputs of the encoder model are fed as…
Jared Frazier
  • 413
  • 1
  • 4
  • 10
0
votes
1 answer

Why can't I perform gradients on a variable passed as an argument to a tf.function?

My training loop was giving me the following warning: WARNING:tensorflow:Gradients do not exist for variables ['noise:0'] when minimizing the loss. After some tinkering I determined this only happened when the noise variable was being passed as an…
partyphysics
  • 175
  • 7
0
votes
1 answer

How to print gradient values for a particular layer of my tensorflow model?

I am training a model in TensorFlow 2.3 I want to see the value of gradients for different layers of my model. I know how to access the layers of my model. with tf.GradientTape() as tape: lstm = model(song_emb_inp=song_emb_id_x_batch, …
n0obcoder
  • 649
  • 8
  • 24
0
votes
0 answers

using tf.gradienttape, loss = mse or huber or cross-entropy, y_true=constant, y_pred=my_network

1. Actor critic model using tf.gradienttape, loss_function = mse or huber or cross-entropy y_true=constant, y_pred=my_network_output, e.g. y_pred = my_netword(input) e.g. loss_actor = tf.losses.MSE(y_true, y_pred) or other such of things,…
Yu Bo
  • 1
  • 2
0
votes
1 answer

Training a basic TensorFlow Model using the GradientTape

Simply for education purposes, I was trying to build upon the Basic training loops tutorial from the TensorFlow homepage to create a simple neural network that classifies points in the plane. So, I have some points in [0,1]x[0,1] stored in a tensor…
Tha_X
  • 13
  • 3
0
votes
1 answer

If-Else Statement in Custom Training Loop in Tensorflow

I created a model class which is a subclass of keras.Model. While training the model, I want to change the weights of the loss functions after some epochs. In order to do that I created boolean variables to my model indicating that the model should…
Burak Bekci
  • 23
  • 6
1 2 3
…
9
10