1

I know how to select which of the tf.Variables I create are subject to training: I call the optimizer with var_list=[]. My problem arises when I don't actually create the variables myself, and thus can't pass them explicitly to the optimizwer. Variables to be optimized are being created for me with the convenience function for a convolution: it will create Weight and Bias Variables and then add these to the trainable_variables() list. 

 x = tf.layers.conv2d(X, filters=64, kernel_size=4, strides=2, padding='same', activation=activation)

However I'm programming a GAN, so I need to alternate between training two sets of variables, one for the generator and one set for the discriminator. Thus I don't want to just train on the entire set of variables in tf.trainable_variables().

I'd like to use the convenience function to set up the model but then get handles to the variables in the model so I can choose which get passed to the optimizer.

Here's an example of the code for the generator. the dicriminator is analgous.

with tf.variable_scope("encoder", reuse=None):
        X = tf.reshape(X_in, shape=[-1, 28, 28, 1])
        x = tf.layers.conv2d(X, filters=64, kernel_size=4, strides=2, padding='same', activation=activation)
        x = tf.nn.dropout(x, keep_prob)
        x = tf.layers.conv2d(x, filters=64, kernel_size=4, strides=2, padding='same', activation=activation)
        x = tf.nn.dropout(x, keep_prob)
        x = tf.layers.conv2d(x, filters=64, kernel_size=4, strides=1, padding='same', activation=activation)
        x = tf.nn.dropout(x, keep_prob)
        x = tf.contrib.layers.flatten(x)
Alexander Hamilton
  • 409
  • 3
  • 14
  • You might be able to look up the variables by name, though I don't know what to look for specifically. Otherwise, you might want to just define the convolutional layer yourself, it's not many lines of code and there are many examples out there. I think layers are for the simple cases. – David Parks Jan 29 '18 at 02:55
  • Thanks David. I agree I could roll it myself, but I'm trying to use the higher level functions. The whole thing just is aching to have some way to do this, since what I'm asking for is pretty basic. Here's what I would have sort of expected. Rather than returning the output ('x' here), it would return an object. then x.out would be the normal output and x.vars would be the trainable vars it created. I'm still betting there is some interface I'm overlooking. – Alexander Hamilton Jan 29 '18 at 04:03

1 Answers1

2

I think I just found the answer. apparently the trick here is to use the "name" and "reuse" attributes of the call to conv2d. If you do this then the name of the variables is just /kernel and /bias.

The follow-up problem is that tf will number these, so if you re-run the algortithm (in ipython or jupyter notebooks for example) you'll have multiple identically named variables. So you have to invoke 'reuse=True' so they get the default name :0 

conv_layer = tf.layers.conv2d(..., name='YOUR_NAME', reuse=True...)

Then your Access the named variables like this: 

gr = tf.get_default_graph() 
conv1_kernel_val = gr.get_tensor_by_name('YOUR_NAME/kernel:0').eval()
conv1_bias_val = gr.get_tensor_by_name('YOUR_NAME/bias:0').eval()

that's not a pretty solution. bad tensorflow, bad dog! You should not have to guess or control the name. There should have been a way to definitively access the variables provided by the call.

hat tip to this post: How to access kernel variables in tf.layers.conv2d?

Alexander Hamilton
  • 409
  • 3
  • 14