3

I used tf.keras to build a fully-connected ANN, "my_model". Then, I'm trying to minimize a function f(x) = my_model.predict(x) - 0.5 + g(x) using Adam optimizer from TensorFlow. I tried the below code:

x = tf.get_variable('x', initializer = np.array([1.5, 2.6]))
f = my_model.predict(x) - 0.5 + g(x)
optimizer = tf.train.AdamOptimizer(learning_rate=.001).minimize(f) 
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(50):
        print(sess.run([x,f]))
        sess.run(optimizer)

However, I'm getting the following error when my_model.predict(x) is executed:

If your data is in the form of symbolic tensors, you should specify the steps argument (instead of the batch_size argument)

I understand what the error is but I'm unable to figure out how to make my_model.predict(x) work in the presence of symbolic tensors. If my_model.predict(x) is removed from the function f(x), the code runs without any error.

I checked the following link, link where TensorFlow optimizers are used to minimize an arbitrary function, but I think my problem is with the usage of underlying keras's model.predict() function. I appreciate any help. Thanks in advance!

NoName
  • 51
  • 1
  • 11

1 Answers1

2

I found the answer!

Basically, I was trying to optimize a function involving a trained ANN w.r.t the input variables to the ANN. So, all I wanted was to know how to call my_model and put it in f(x). Digging a bit into the Keras documentation here: https://keras.io/getting-started/functional-api-guide/, I found that all Keras models are callable just like the layers of the models! Quoting the information from the link,

..you can treat any model as if it were a layer, by calling it on a tensor. Note that by calling a model you aren't just reusing the architecture of the model, you are also reusing its weights.

Meanwhile, the model.predict(x) part expects x to be numpy arrays or evaluated tensors and does not take tensorflow variables as inputs (https://www.tensorflow.org/api_docs/python/tf/keras/Model#predict).

So the following code worked:

## initializations
sess = tf.InteractiveSession()
x_init_value = np.array([1.5, 2.6])
x_placeholder =  tf.placeholder(tf.float32)
x_var = tf.Variable(x_init_value, dtype=tf.float32)

# Check calling my_model
assign_step = tf.assign(x_var, x_placeholder)
sess.run(assign_step, feed_dict={x_placeholder: x_init_value})
model_output = my_model(x_var) # This simple step is all I wanted!
sess.run(model_output) # This outputs my_model's predicted value for input x_init_value

# Now, define the objective function that has to be minimized
f = my_model(x_var) - 0.5 + g(x_var) # g(x_var) is some function of x_var

# Define the optimizer
optimizer = tf.train.AdamOptimizer(learning_rate=.001).minimize(f) 

# Run the optimization steps
for i in range(50): # for 50 steps
    _,loss = optimizer.minimize(f, var_list=[x_var])
    print("step: ", i+1, ", loss: ", loss, ", X: ", x_var.eval()))
NoName
  • 51
  • 1
  • 11