19

Given I have a linear model as the following I would like to get the gradient vector with regards to W and b.

# tf Graph Input
X = tf.placeholder("float")
Y = tf.placeholder("float")

# Set model weights
W = tf.Variable(rng.randn(), name="weight")
b = tf.Variable(rng.randn(), name="bias")

# Construct a linear model
pred = tf.add(tf.mul(X, W), b)

# Mean squared error
cost = tf.reduce_sum(tf.pow(pred-Y, 2))/(2*n_samples)

However if I try something like this where cost is a function of cost(x,y,w,b) and I only want to gradients with respect to w and b:

grads = tf.gradients(cost, tf.all_variable())

My placeholders will also be included (X and Y). Even if I do get a gradient with [x,y,w,b] how do I know which element in the gradient that belong to each parameter since it is just a list without names to which parameter the derivative has be taken with regards to?

In this question I'm using parts of this code and I build on this question.

dga
  • 21,757
  • 3
  • 44
  • 51
user3139545
  • 6,882
  • 13
  • 44
  • 87

1 Answers1

36

Quoting the docs for tf.gradients

Constructs symbolic partial derivatives of sum of ys w.r.t. x in xs.

So, this should work:

dc_dw, dc_db = tf.gradients(cost, [W, b])

Here, tf.gradients() returns the gradient of cost wrt each tensor in the second argument as a list in the same order.

Read tf.gradients for more information.

buydadip
  • 8,890
  • 22
  • 79
  • 154
Priyatham
  • 2,821
  • 1
  • 19
  • 33