In this(https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/eager/python/g3doc/guide.md) tutorial, a method of implementing custom gradient function is provided.
@tfe.custom_gradient
def log1pexp(x):
e = tf.exp(x)
def grad(dy):
return dy * (1 - 1 / (1 + e))
return tf.log(1 + e), grad
grad_log1pexp = tfe.gradients_function(log1pexp)
# Works as before at x = 0.
assert 0.5 == float(grad_log1pexp(0.)[0])
# But now works at x = 100 as well.
assert 1.0 == float(grad_log1pexp(100.)[0])
I would like to implement a gradient function for matrix exponential, in which case the variable is a tensor. I tried the straight forward approach but it seems failed.
My questions are:
- What's the meaning of dy here?
- How should these code be adopted to tensors?
