0

I have a keras model with a two-dimensional output (binary classification).

model.output # <tf.Tensor 'dense_1_3/MatMul:0' shape=(?, 2) dtype=float32>

and 

model.input # <tf.Tensor 'bidirectional_1_input:0' shape=(?, ?, 200) dtype=float32>

I evaluated three different gradients for some example input of shape (1,50,200)

gradients0 = K.gradients(model.output[:,0] model.inputs)
gradients1 = K.gradients(model.output[:,1], model.inputs)
gradients2 = K.gradients(model.output, model.inputs)

I thought, the first two expressions yield the gradient for the single output neurons and the last one yields a tensor containing the first two expressions. To my surprise, all three gradients have a shape of (1,50,200). In my opinion, gradients2 needs to be of shape (2,50,200) since model.output is two dimensional. What is keras computing in this case?

AlexConfused
  • 801
  • 1
  • 10
  • 15

1 Answers1

2

Keras.backend.gradients() expects the output to be a scalar function, not a multi-dimensional one. I've found with a small example that K.gradients() performs identically that tf.gradients(). This way (as seen here: https://www.tensorflow.org/api_docs/python/tf/gradients), your gradients2 is returning a list of Tensor of length len(xs) where each tensor is the sum(dy/dx) for y in ys, which explains why the first shape dimension is 1 and not 2.

This link can help you: Tensorflow gradient with respect to matrix

Alberto Martín
  • 90
  • 6