0

I am trying to understand the purpose of TF GRADIENT TAPE, in the following code:

import tensorflow as tf
var = tf.Variable(5.0)
with tf.GradientTape() as tape:
    op = (2*var)+(var*var)
    diff = tape.gradient(op,var)
print (diff)

Op:

diff = tf.Tensor(12.0, shape=(), dtype=float32)

I am confused because since var=5, the op=(2*5)+(5*5)=>35, and if I am calculating the derivative of a constant then diff should be 0

I understand the reason its 12, because its not taking the var as 5 instead (2*var)+(var*var)=> 2var+var**2 so calculating the derivative of this function becomes 2+2*var=>12.

But what I dont understand is, why the value given for var is not considered?

data_person
  • 4,194
  • 7
  • 40
  • 75
  • Sorry what do you mean the value given for `var` is not considered? The gradient `2+2*var` is clearly a function of `var`. – rchome Nov 26 '21 at 00:24
  • @rchome if the given value of ```var``` is considered in the ```tape``` then the derivative should be ```0``` right? – data_person Nov 26 '21 at 00:28
  • 1
    `var` is a variable, not a constant. The derivative of `op` with respect to `var` is 12 in this case. If you instead used `var = tf.constant(5.0)`, you would get `None` for the gradient since the `GradientTape` is not "watching" `var`. https://www.tensorflow.org/guide/autodiff#controlling_what_the_tape_watches – rchome Nov 26 '21 at 00:33

1 Answers1

1

It is producing the right result.
tf.GradientTape context that will automatically record every operation that involves a variable. If you consider the partial derivative for the below function with regard to var

def v(var):
  return 2 * var + var * var

It will be 2 + 2*var, when the value of var=5 it will be 2 + 2*5 which is 12 returned by the gradient below.

var = tf.Variable(5.0)
with tf.GradientTape() as tape:
  z = v(var)
diff = tape.gradient(z,var)
print (diff)
tf.Tensor(12.0, shape=(), dtype=float32)

Try changing the variable to different values, for the function v, the equation 2 + 2*var holds good.