Questions tagged [gradienttape]
147 questions
2
votes
1 answer
How to find the analytical gradient using tensorflow gradienttape
Suppose we have some function
y=x^2
We can then use gradient tape to automatically calculate the gradient for us (when we provide some values of x to tensorflow
x = tf.Variable(3.0)
with tf.GradientTape() as tape:
y = x**2
dy_dx =…
Ting Wai Au
- 31
- 3
2
votes
0 answers
Gradient of a neuronal network
I have a trained neural network that predicts the noise of a given image.
Now I want to use it to calculate a subgradient of my NN wrt the norm of the output.
I want to use this in a larger algorithm, but since I can not get it to work as expected,…
Sumny
- 95
- 5
2
votes
1 answer
Use of tf.GradientTape() exhausts all the gpu memory, without it it doesn't matter
I'm working on Convolution Tasnet, model size I made is about 5.05 million variables.
I want to train this using custom training loops, and the problem is,
for i, (input_batch, target_batch) in enumerate(train_ds): # each shape is (64, 32000, 1)
…
HyeonPhil Youn
- 428
- 4
- 11
2
votes
0 answers
tensorflow 2, gradients are an empty list
I'm reorganizing my code to be easy to read, but when compiling it says:
ValueError: No gradients provided for any variable: ['enc_conv_4/kernel:0'.... I know my loss function is differentiable cuz the code worked before touch it, but now is missing…
SebastianRL
- 23
- 4
2
votes
0 answers
how to visualize weights and bias in tensorboard when using tf.GradientTape() in TensorFlow 2.3.0
Using tensorflows tutorial on DCGAN as an example:
https://www.tensorflow.org/tutorials/generative/dcgan?hl=en
To log the loss, the following example was used:
https://www.tensorflow.org/tensorboard/get_started?hl=en
Using the above as a reference,…
trennerzz
- 21
- 2
2
votes
0 answers
How to debug exploding gradient (covariance matrix) in Tensorflow 2.0 (TFP)
A question that comes from the fact that I never had to debug my models in TF so deeply.
I'm running a variational inference with a full-rank Gaussian approximation using Tensorflow Probability. I noticed my optimization often explodes. Here is my…
owy
- 21
- 5
2
votes
0 answers
"Shapes of all inputs must match" error loss function when trying to do custom training with tf.GradientTape()
I'm using Python 3.7.7. and Tensorflow 2.1.0 with Functional API and Eager Execution.
I'm trying to do custom training, with an encoder extracted from a U-Net pretrained network:
I get the U-Net model without compile it.
I have loaded the weights…
VansFannel
- 45,055
- 107
- 359
- 626
2
votes
1 answer
TF2 - GradientTape vs Model.fit() - Why does GradientTape doesn't work?
Good Evening,
I want to implement a toy example for a simple regression problem with tf2 and the Gradient Tape function. With Model.fit it learns properly but the same with the GradientTape does something but the loss doesn't move compared to…
2
votes
1 answer
tf.GradientTape() doesn't work on sliced outputs
Here is a piece of code which I tried to run:
import tensorflow as tf
a = tf.constant([[1, 2], [2, 3]], dtype=tf.float32)
b = tf.constant([[1, 2], [2, 3]], dtype=tf.float32)
with tf.GradientTape() as tape1, tf.GradientTape() as tape2:
…
Akshat Agrawal
- 31
- 3
2
votes
1 answer
Out of memory OOM using tensorflow gradient tape but only happens when I append a list
I've been working on a data set (1000,3253) using a CNN. I'm running gradient calculations through gradient tape but it keeps running out of memory. Yet if I remove the line appending a gradient calculation to a list the script runs through all the…
nauge
- 23
- 3
2
votes
1 answer
Problem computing partial derivatives with GradientTape() in TensorFlow2
i have problems in the computation of gradients using automatic differentiation in TensorFlow. Basically i want to create a neural network which has just one output-value f and get an input of two values (x,t). The network should act like a…
MichiTrain
- 75
- 6
2
votes
1 answer
Tensorflow GradientTape does not trace optimizer.apply_gradients?
import tensorflow as tf
def f(x):
return tf.multiply(x, x)
x = tf.Variable([3.])
with tf.GradientTape() as test_tape:
test_tape.watch(x)
with tf.GradientTape() as train_tape:
train_tape.watch(x)
fx = f(x)
…
Gu Liqi
- 41
- 3
2
votes
0 answers
Tensorflow 2 gradient tape not working as expected
I am currently training a large object detection model in Tensorflow 2 with a custom training loop using gradient tape. The problem is that the model is not improving the loss as the gradients are very low. I reproduced the problem on a simple…
Maximilian
- 21
- 2
2
votes
1 answer
Tensorflow 2.0 Autograph indirect modification (hidden states) works, when it shouldn't
So, here it says that indirect modification should not work, which means that changes would be invisible (What does invisible change mean anyway?)
But this code computes the gradient correctly:
import tensorflow as tf
class C:
def…
Feri
- 1,071
- 7
- 19
2
votes
1 answer
Gradient computations in Tensorflow 2.0
Here is my example from Tensorflow 2.0:
import tensorflow as tf
w = tf.Variable([[1.0]])
with tf.GradientTape() as tape_1:
loss_1 = w * w
with tf.GradientTape() as tape_2:
loss_2 = w * w * w
grad_1 = tape_1.gradient(loss_1, w)
grad_2 =…
user1700890
- 7,144
- 18
- 87
- 183