Stack Overflow
Stack Overflow

Questions tagged [gradienttape]

147 questions
1
vote
0 answers

TensorFlow RuntimeError: "Attempting to capture an EagerTensor without building a function"

I am trying to build a neural network in Python for solving PDEs, and, as such, I have had to write custom training steps. My training function looks like this: ... tf.enable_eager_execution() class PDENet: ... def train_step(): …
Isaac Krementsov
  • 646
  • 2
  • 12
  • 28
1
vote
0 answers

Computing gradients for outputs taken from intermediate layers and updating weights using optimizer

I am trying to implement below architecture and not sure in applying gradient tape properly. In the above architecture we can see, outputs taken from multiple layers in the blue boxes. Each blue box is termed as loss branch in the paper which…
Looters
  • 61
  • 3
1
vote
1 answer

Why the gradients are unconnected in the following function?

I am implementing a customer operation whose gradients must be calculated. The following is the function: def difference(prod,box): result = tf.Variable(tf.zeros((prod.shape[0],box.shape[1]),dtype=tf.float16)) for i in…
Shanthan K
  • 87
  • 5
1
vote
1 answer

Why is batch_size being multiplied to GradientTape results in Tensorflow?

I'm trying to get the gradients of a loss function w.r.t to another tensor. But the gradients are being multiplied by input batch size that I feed into my model. import tensorflow as tf from tensorflow.keras import Sequential, layers #Sample States…
Uchiha Madara
  • 984
  • 5
  • 16
  • 35
1
vote
1 answer

Tensorflow: correct way to initialize concatenated tensors?

Here's a MWE for the problem I'm facing: import tensorflow as tf with tf.GradientTape() as tape: x = tf.Variable(0.0) y = tf.Variable(x) z = x print(tape.gradient(y, x)) # None print(tape.gradient(z, x)) # 1.0 Well, obviously this is is…
Abhimanyu Pallavi Sudhir
  • 202
  • 1
  • 12
1
vote
0 answers

Tensorflow tf.GradientTape().gradient returns none

I designed a function to calculate gradients from loss and model.trainable_variables with Tensorflow GardientTape. I used this function to perform split-learning, which means the model is devided and trained on a client up to a specific layer. The…
Melvin
  • 21
  • 2
1
vote
0 answers

Why is Tensorflow's Gradient Tape returning None when trying to find the gradient of loss wrt input?

I have a CNN model built in keras which uses an SVM in its last layer. I get the prediction of this SVM by putting in an input into the CNN model, extracting the relevant features and then putting those features into my SVM to get an output…
MangLion
  • 153
  • 5
1
vote
1 answer

Implementing The 'Learning To Read With Tensorflow' Talk From TF Summit 2020 - EncoderDecoder Seq2Seq Model In Tensorflow 2.1/2.2 - Custom Train Step

Background Info I am creating Google Colabs for each talk I found interesting from the Tensorflow 2020 Summit. As a note, I am using Tensorflow 2.1. I have encountered a problem when attempting to implement the 'Learning To Read With Tensorflow'…
Darien Schettler
  • 546
  • 4
  • 13
1
vote
0 answers

How different is model.fit from using explicit GradientTape on the model.trainable variables?

So I have been experimenting with both Keras' Model.fit() and the low-level TF GradientTape for optimising the trainable parameters of a neural network and noticed that the Keras version is significantly better. Code for the Keras Optimised Version…
Vignesh Gopakumar
  • 143
  • 1
  • 3
  • 7
1
vote
2 answers

Why is GradientTape returning None when I use numpy math

Why is GradientTape returning None when I use numpy math I am trying to understand tensorflow GradientTape calculation for RL loss function. When I call a function using np.math the GradientTape returns None. If I use tf.math in the function it…
Ulf Wållgren
  • 161
  • 2
  • 8
1
vote
0 answers

Tensorflow gradientTape gives different result when calculating the same gradient twice

I'm experimenting with TF 2.0. I want to record the gradient and weights norm across my NN. To do so I'm using the following code. def get_weights_norm(layer, optim_iters, log=False): """ Calculate norm of layer's weights and save it as…
Mick Hardins
  • 25
  • 1
  • 9
1
vote
2 answers

Exploding LOSS in TensorFlow 2.0 Linear Regression Example using GradientTape

I'm trying to construct a little educational example for multivariate linear regresssion, but the LOSS is increasing until it explodes rather than getting smaller, any idea? import tensorflow as tf tf.__version__ import numpy as np data =…
Romeo Kienzler
  • 3,373
  • 3
  • 36
  • 58
0
votes
0 answers

tf.keras.optimizers.Adam.apply_gradients triggers tf.function retracing

I'm getting a memory leak and I believe it to be linked to the following warning: WARNING:tensorflow:6 out of the last 6 calls to triggered tf.function retracing. Tracing is expensive and…
Bryan Carty
  • 83
  • 6
0
votes
0 answers

How to perform gradient descent on a keras neural network with two outputs

I have a network that takes two inputs, the first input is passed through a series of convolutional layers, before being flattened and concatenated with the second input. The result is then passed through a number of dense layers. The second portion…
Bryan Carty
  • 83
  • 6
0
votes
0 answers

How to get gradients from GradientTape

I wanted to get the gradients from the gradient tape so in tf version 1 I used something like - grads = model.optimizer.get_gradients(model.total_loss, model.trainable_weights) symb_inputs = (model._feed_inputs + model._feed_targets +…
Shashank Priyadarshi
  • 1
  • 1
1 2 3
…
9 10