2

An example is the following: The goal is to optimize a function f(x1,..,xN) subject to the constraint x1 + .. + xN = 1 with 0 < x_i <= 1. How do I do this with tensorflow in general (with arbitrary constraints)?

I have seen the ScipyOptimizerInterface which allows to do this.

In this special case, a different approach is to use the nnsoftmax transform to write x_i = exp(y_i)/sum_i(exp(y_i)), i=1,..,N. y1,..,yN are unconstrained now and can be optimized with SGD, i.e. with TensorFlow without any constraints. In GPFlow, this would be equivalent to a transform associated to the variables {y1,..,yN}.

Is there any other approach?

Matthew Strawbridge
  • 19,940
  • 10
  • 72
  • 93
Maximilian Mordig
  • 1,333
  • 1
  • 12
  • 16
  • This feels broad. In general, with arbitrary constraints, you will need some NLP-solver (which are mostly Interior-point based; quite complex software) like Ipopt (where the final question then is: how transfer tf's differentiation capabilities to ipopt's core). There is much tf-free software doing autodiff (mathematical-programming; but mostly based on full-batch opt). The simple example allows *projection on the probability-simplex* and as this projection is cheap (linear-time if done right) one approach is: projected (stochastic) gradient (own optimizer not hard to implement) – sascha Mar 09 '18 at 16:02

0 Answers0