0

Typically a tf optimizer flow is as follows:

# Create an optimizer.
opt = GradientDescentOptimizer(learning_rate=0.1)

# Compute the gradients for a list of variables.
grads_and_vars = opt.compute_gradients(loss, <list of variables>)

# grads_and_vars is a list of tuples (gradient, variable).  Do 
# whatever you
# need to the 'gradient' part, for example cap them, etc.
capped_grads_and_vars = [(MyCapper(gv[0]), gv[1]) for gv in grads_and_vars]

# Ask the optimizer to apply the capped gradients.
opt.apply_gradients(capped_grads_and_vars)

How can I modify this flow so as loss function is computed using different datapoints after each optimizer step and these datapoints are computed as a function of the updated parameters?

Say that I have a stochastic process which has a certain parameterization and parameter values are learned by gradient descent. After updating the parameter values, I have to resample points from this process and evaluate the likelihood on these points (instead of the points I used in my previous step). Can I easily incorporate this in a typical tf optimizer? Conceptually, is similar to batch gradient descent, however datapoints of each batch are generated over the gradient descent loop in a way that depends on the parameter values of the previous iteration.

Dion
  • 123
  • 2
  • 8
  • Regarding the process by which you sample new points to be used in the loss function: Can you easily represent this process in a Tensorflow graph, or does it need to be written in Python? If it can be easily made into a graph, you can construct the resampling step, hook it up to one or more `tf.assign` assigning into variables (marked non-trainable), and use the variables as inputs to your loss function. – nanofarad Dec 29 '18 at 15:45
  • thanks for your reply. Resampling is complicated and has been written in Python at the moment – Dion Dec 29 '18 at 16:36
  • would switching to pytorch make this particular problem easier to address? – Dion Dec 29 '18 at 16:54
  • I am not familiar with Pytorch to any extent, so I'm unsure. – nanofarad Dec 29 '18 at 16:55
  • I don't have to keep track of the gradient in the simulation process - is this simplifying things? I just have to plug in the value of the parameters (which are tf.variables) and generate a list of data. Then use these datapoints as placeholders for the new evaluation of the likelihood – Dion Jan 09 '19 at 11:42

0 Answers0