1

I have an encoder decoder network mimicking the one produced in this tutorial: https://towardsdatascience.com/how-to-implement-seq2seq-lstm-model-in-keras-shortcutnlp-6f355f3e5639

However the output of the decoder LSTM will be numbers between 0 and 1. However the words were tokenized in this tutorial to be integers. How do I convert this output between 0 and 1 back to words using this tokenizing?

The other option could be to use the one hot encoding tokenization but surely you'd still have to round the output to turn the floating outputs to integers?

Haztec
  • 31
  • 3

2 Answers2

1

Use tf.math.argmax() to find index of the best word. And then extract the word from your vocabulary

Andrey
  • 5,932
  • 3
  • 17
  • 35
  • Thanks for the reply. I'm not sure how that would work though - What do you mean by "best word"? – Haztec Nov 10 '20 at 21:44
  • Softmax gives you a probability distribution between word indices. The more value - the better the word – Andrey Nov 11 '20 at 07:03
1

The tutorial you quote uses a vocabulary of 15000 tokens. It outputs a sequence of 10 tokens per sample such that each of the 10 words is a vector of 15000 floats. You can think of these as a probability such that ideally the sum of these 15000 floats adds up to one and the highest score tells you which token was chosen. So on input each token gets converted into a 1 hot encoded vector of size 15000 and on output you have a softmax node that outputs a score per token where the highest score give you the selected token.

Pedro Marques
  • 2,642
  • 1
  • 10
  • 10