1

I would like to understand how/why name_scope and variable_scope are both used in the code below. I have looked at the Tensorflow documentation for these methods and I have also looked at this very helpful SO post: Difference between variable_scope and name_scope in TensorFlow but that has not cleared up my confusion.

I am trying to reuse some code from this paper, and in particular I'm looking at how they coded up an autoencoder. input_param is a tensor passed in to a function, and the code is inside the function.

with tf.name_scope("input") as sc:
    frednet=input_param

mk_lstm = lambda x: [rnn.LSTMCell(12, use_peepholes=True,
                    initializer=layers.xavier_initializer()) for i in range(x)]
with tf.variable_scope("lstm1") as sc:

    #lstm=rnn.LSTMCell(5,use_peepholes=True,initializer=layers.xavier_initializer())
    lstm=rnn.MultiRNNCell(mk_lstm(2))
    frednet,st=tf.nn.dynamic_rnn(lstm,frednet,dtype=tf.float32)

with tf.variable_scope("lstm2") as scb:  
    #lstm=rnn.LSTMCell(12,use_peepholes=True,initializer=layers.xavier_initializer())
    lstm=rnn.MultiRNNCell(mk_lstm(2))
    frednet,st=tf.nn.dynamic_rnn(lstm,frednet,dtype=tf.float32,initial_state=st)

net=sm.flatten(frednet[:,:,:])
#     net=sm.dropout(net,keep_prob=.9)
net=sm.fully_connected(net,num_outputs=1,weights_initializer=layers.xavier_initializer(),
                           weights_regularizer=sm.l2_regularizer(1e-7))

This code works fine BUT I would like to understand why modifications I made caused errors:

(1) if I delete the name_scope portion of the code then my model doesn't really train and keeps printing out zeros.

(2) If I also delete the variable_scope portion of the code, then my code won't even run as tensorflow throws some name conflict error.

Here are my questions:

(1) What is going on with error (1) described above? How this name_scope help with the "sharing" of the frednet variable?

(2) Is there a reason they used first sc and then scb for the variable scope (first one matches the name scope but second doesn't). My understanding of Python is that these names are scoped only within the with clause so why don't all 3 just use the same?

(3) For error (2), is it correct that this is happening because MultiRNNCell has default names that it uses, and these are in conflict if I call that function multiple times? Is it always necessary to use variable_scope in the event of multiple uses of MultiRNNCell?

Thanks.

TFdoe
  • 571
  • 5
  • 16

0 Answers0