12

I am confused about what dynamic RNN (i.e. dynamic_rnn) is. It returns an output and a state in TensorFlow. What are these state and output? What is dynamic in a dynamic RNN, in TensorFlow?

nbro
  • 15,395
  • 32
  • 113
  • 196
xlax
  • 2,661
  • 6
  • 12
  • 15

2 Answers2

16

Dynamic RNN's allow for variable sequence lengths. You might have an input shape (batch_size, max_sequence_length), but this will allow you to run the RNN for the correct number of time steps on those sequences that are shorter than max_sequence_length.

In contrast, there are static RNNs, which expect to run the entire fixed RNN length. There are cases where you might prefer to do this, such as if you are padding your inputs to max_sequence_length anyway.

In short, dynamic_rnn is usually what you want for variable length sequential data. It has a sequence_length parameter, and it is your friend.

nbro
  • 15,395
  • 32
  • 113
  • 196
AlexDelPiero
  • 289
  • 1
  • 8
  • 1
    what is state and output? can you explain? – xlax Apr 04 '17 at 05:37
  • @xlax A pretty good explanation and example of the outputs can be found here: https://stackoverflow.com/a/48239320/8236733 – lampShadesDrifter Sep 10 '18 at 21:51
  • Looks like `sequence_length` is a vector specifying the length in each instance/batch. But ideally, the RNN should be able to infer the lengths from, well, the lengths (the input could be a list/array of variable-length lists/arrays), so you don't need to specify `sequence_length`. I think the keras LSTM handles this nicely, but I'm not sure about https://www.tensorflow.org/api_docs/python/tf/nn/dynamic_rnn. – flow2k Aug 22 '19 at 20:28
2

While AlexDelPiero's answer was what I was googling for, the original question was different. You can take a look at this detailed description about LSTMs and intuition behind them. LSTM is the most common example of an RNN.

http://colah.github.io/posts/2015-08-Understanding-LSTMs/

The short answer is: the state is an internal detail that is passed from one timestep to another. The output is a tensor of outputs on each timestep. You usually need to pass all outputs to the next RNN layer or the last output for the last RNN layer. To get the last output you can use output[:,-1,:]

alyaxey
  • 1,129
  • 12
  • 10