1

Trying to initialize my seq2seq-model. I am using the GRU Cells, but cannot figure out Xavier-Initialization. The code in rnn_cell.py doesnt seem to allow that. Any ideas? Need to do this manually?

thx

Phillip Bock
  • 1,879
  • 14
  • 23

1 Answers1

2

The cell's weights are created using tf.get_variable() without specifying an initializer parameter. From https://www.tensorflow.org/versions/r0.10/api_docs/python/state_ops.html#get_variable:

If initializer is None (the default), the default initializer passed in the variable scope will be used.

Therefore, something like the following should work:

cell = tf.nn.rnn_cell.GRUCell(256)
with tf.variable_scope('RNN', initializer=tf.contrib.layers.xavier_initializer()):
    outputs, state = tf.nn.dynamic_rnn(cell, ...)
fwalch
  • 1,136
  • 11
  • 25