1

I just have a short question regarding the pytorch version of OpenNMT. There does not seem to be an option to return the hidden state of encoder and decoder in the options. Am I missing a flag or is this not an option in OpenNMT-py?

thaumoctopus
  • 113
  • 1
  • 13

1 Answers1

1

What do you mean by the encoder and decoder does not return hidden state?

If you see the RNNEncoder, it returns encoder_final, memory_bank, lengths where the memory_bank represents the hidden state which is of shape seq_len x batch_size x hidden_size. And the encoder_final is in general used by the decoder in a sequence-to-sequence model.

Now, let's see the RNNDecoder. As we see, the forward() method returns a FlaotTensor and a dictionary of FlaotTensors.

(FloatTensor, dict[str, FloatTensor]):
* dec_outs: output from the decoder (after attn)
  ``(tgt_len, batch, hidden)``.
* attns: distribution over src at each tgt
  ``(tgt_len, batch, src_len)``.

Usually, we use the dec_outs in sequence-to-sequence tasks. For example, in natural language generation tasks, we feed the dec_outs to a softmax layer to predict tokens.

There are several other types of encoders/decoders which you can explore from the following two links.

Wasi Ahmad
  • 35,739
  • 32
  • 114
  • 161
  • In the memory_bank, how can we distinguish cell state from hidden state for an LSTM model? I don't get a separate value for the cell. – thaumoctopus Jun 26 '19 at 22:49