LSTM Dropout Wrapper Rank Error

Question

I have an LSTM model which works "correctly" when specifying my dropout keep rate as follows:

layers = [tf.contrib.rnn.DropoutWrapper(tf.contrib.rnn.BasicLSTMCell(num_units=n_neurons, activation=tf.nn.tanh), output_keep_prob=0.5)
      for layer in range(n_layers)]

But naturally, I want to make the output_keep_prob into a float variable that I can change when I am in Train vs. Test. I have done this as below

    output_keep_prob = tf.placeholder_with_default(1.0, tf.float32)
...
layers = [tf.contrib.rnn.DropoutWrapper(tf.contrib.rnn.BasicLSTMCell(num_units=n_neurons, activation=tf.nn.tanh), output_keep_prob=output_keep_prob)
          for layer in range(n_layers)]
...
sess.run(training_op, feed_dict={X: x_batch, y: y_batch, output_keep_prob: 0.5})

However, Tensorflow is throwing off errors when I do this:

ValueError: Shapes must be equal rank, but are 0 and 1 for 'PlaceholderWithDefault' (op: 'PlaceholderWithDefault') with input shapes: [].

I think I may need to specify different dimensions on the placeholder, but I haven't encountered this problem on standard feed-forward dropout. I've tried a few variations of specifying n_layers in the dimensionality (which is what I think I need to fix the issue?) but haven't had any success.

score 0 · Accepted Answer · answered Apr 06 '18 at 18:25

Easier fix than I thought ...

Appears that I need to have my "default" input as a tensor rather than a scalar, so

output_keep_prob = tf.placeholder_with_default([1.0], tf.float32)

is the first step, and in my declarations for my test/train runs, I need to use the same approach, denoting the keep rate with [0.5] instead of just 0.5.

sess.run(training_op, feed_dict={X: x_batch, y: y_batch, output_keep_prob:[0.5]})

LSTM Dropout Wrapper Rank Error

1 Answers1