2

I am wondering why I am getting none for my grads in the following code:

import tensorflow.keras.losses as losses
loss = losses.squared_hinge(y_true, y_pred)

from tensorflow.keras import backend as K
grads = K.gradients(loss, CNN_model.input)[0]
iterate = K.function([CNN_model.input], [loss, grads])

my CNN_model.input is: <tf.Tensor 'conv2d_3_input:0' shape=(?, 28, 28, 1) dtype=float32>

my loss is: <tf.Tensor 'Mean_3:0' shape=(1,) dtype=float64>

Note: I am passing the predicted output of an SVM as y_pred for my application if that is of importance.

MangLion
  • 153
  • 5

1 Answers1

2

As far as I understood from my previous experience, Tensorflow needs to use GradientTape in order to record the activity of a certain variable and so to compute its gradients. In your case should be something like that:

x = np.random.rand(10) #your input variable
x = tf.Variable(x) #to be evaluated by GradientTape the input should be a tensor
with tf.GradientTape() as tape:
    tape.watch(x) #with this method you can observe your variable
    proba = model(x) #get the prediction of the input
    loss = your_loss_function(y_true, proba) #compute the loss

gradient = tape.gradient(loss, x) #compute the gradients, this must be done outside the recording
Davide Giordano
  • 68
  • 5
  • is gradient tape available in tensorflow 1? – MangLion Apr 17 '20 at 01:52
  • @MangLion it should be present also in Tensorflow 1. Unfortunately I'm not sure so it has to be verified. – Davide Giordano Apr 17 '20 at 05:26
  • Regardless, this was quite useful thanks. One final thing,just to confirm. In your example, you calculated the gradient wrt to the input right? – MangLion Apr 17 '20 at 10:21
  • @MangLion yes, sure! You can see that because I've used the prediction of the input to calculate the loss. – Davide Giordano Apr 17 '20 at 11:57
  • Hi again, I have another question still related to this problem of getting None after switching to this method, which you might be interested in posted here: https://stackoverflow.com/questions/61367182/why-is-tensorflows-gradient-tape-returning-none-when-trying-to-find-the-gradien if not, no worries. – MangLion Apr 22 '20 at 14:11