Stack Overflow
Stack Overflow

Questions tagged [sequence-to-sequence]

This tag is used for Google's deprecated seq2seq framework, an encoder-decoder framework for Tensorflow (revamped version is called Neural Machine Translation)

94 questions
2
votes
1 answer

Seq2Seq Models for Chatbots

I am building a chat-bot with a sequence to sequence encoder decoder model as in NMT. From the data given I can understand that when training they feed the decoder outputs into the decoder inputs along with the encoder cell states. I cannot figure…
Subham Mukherjee
  • 779
  • 1
  • 7
  • 13
2
votes
1 answer

Training trained seq2seq model on additional training data

I have trained a seq2seq model with 1M samples and saved the latest checkpoint. Now, I have some additional training data of 50K sentence pairs which has not been seen in previous training data. How can I adapt the current model to this new data…
Deepak Mishra
  • 25
  • 1
  • 7
2
votes
1 answer

Multiple issues with axes while implementing a Seq2Seq with attention in CNTK

I'm trying to implement a Seq2Seq model with attention in CNTK, something very similar to CNTK Tutorial 204. However, several small differences lead to various issues and error messages, which I don't understand. There are many questions here, which…
Skiminok
  • 2,801
  • 1
  • 24
  • 29
2
votes
0 answers

Seq2seq multiple input features (Passing multiple word/word tokens as input)

Is there a way to pass extra feature tokens along with the existing word token (training features/source file vocabulary) and feed it to the encoder RNN of seq2seq?. Since, it currently accepts only one word token from the sentence at a time. Let me…
siv
  • 31
  • 5
2
votes
1 answer

Tensorflow NotFoundError

I'm running a custom code to train my own Seq2Seq model on tensorflow. I'm using multi-rnn cells and embedding_attention_seq2seq. While restoring the model I get the following error: 2017-07-14 13:49:13.693612: W…
Vidur Satija
  • 21
  • 6
2
votes
1 answer

How to use the schedule sampling in beam search decoder in tensorflow.

The basic decoder contains a parameter to add helper method that can be a schedule sampling helper. But the beam search decoding does not contain any helper parameter. While in the code it looks like there is some sampling used, but it is not clear…
pratsbhatt
  • 1,498
  • 10
  • 20
2
votes
2 answers

How to use AttentionMechanism with MultiRNNCell and dynamic_decode?

I want to create a multi-layered dynamic RNN-based decoder that uses an attention mechanism. To do this, I first create an attention mechanism: attention_mechanism = BahdanauAttention(num_units=ATTENTION_UNITS, …
Rylan Schaeffer
  • 1,945
  • 2
  • 28
  • 50
2
votes
0 answers

Running of the block with creation of AttentionSeq2Seq in google/seq2seq

Recently Google had released new implementation for seq2seq algorithm: https://github.com/google/seq2seq/blob/master/docs/nmt.md. They had changed the entire internal structure of the code. I tried to modify the code for mine purposes (I created new…
Nurtas
  • 43
  • 5
2
votes
0 answers

How to use tf.contrib.seq2seq.BahdanauAttention

I am trying to produce a simple code for a seq2seq model with attention in tf 1.1. I am not sure what is the parameter "depth of query mechanism ". I am getting an error on creation of Attention Mechanisms saying that: TypeError: int() argument…
E.Asgari
  • 315
  • 1
  • 3
  • 10
2
votes
1 answer

TensorFlow: nr. of epochs vs. nr. of training steps

I have recently experimented with Google's seq2seq to set up a small NMT-system. I managed to get everything working, but I am still wondering about the exact difference between the number of epochs and the number of training steps of a model. If I…
milvala
  • 311
  • 2
  • 13
2
votes
0 answers

Do I need the Sequence Length parameter for RNNCell in Tensorflow

If my input is variable length and I pad it, for example: [X, Y, Z, PAD, PAD] And I pass to Tensorflow's RNN cell that: sequence_length=3 Computation is stopped before the PAD symbol is evaluated, and you get a sequence of output states (one for…
user2827214
  • 1,191
  • 1
  • 13
  • 32
1
vote
0 answers

what should be the value of the "input_lenghts" in "forward function" in encoder rnn in seq2seq model

i'm training an encoder rnn for sequence to sequence model with batches of 10 sentences . every batches have 10 sentences. and every sentences have 60 words. in the encoder network of the seq2seq model what should be the value of the…
mahdi
  • 13
  • 2
1
vote
0 answers

generative question answering huggingface model

Is there an open-source generative question-answering model on huggingface where we can provide a large document as KB (Knowledge Base) such that given a question related to that KB, the model outputs a relevant answer?
user3303020
  • 933
  • 2
  • 12
  • 26
1
vote
0 answers

LSTM to GRU sampling model issue

I am following this (https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html) sequence-to-sequence model tutorial. However, when I try to use GRU instead of LSTM in the model. I got the following error. # GRU #…
David98
  • 31
  • 3
1
vote
2 answers

Concatenate layer shape error in sequence2sequence model with Keras attention

I'm trying to implement a simple word-level sequence-to-sequence model with Keras in Colab. I'm using the Keras Attention layer. Here is the definition of the model: embedding_size=200 UNITS=128 encoder_inputs = Input(shape=(None,),…
Gianni Pinotti
  • 60
  • 6
1 2
3
4 5 6 7