Seq2Seq is a sequence to sequence learning add-on for the python deep learning library.
Questions tagged [seq2seq]
318 questions
3
votes
0 answers
Why does my testing loss increases in my Seq2Seq?
I'm building a Seq2seq neural network. It's a video to natural langage model
The problem:
My training loss decreases normaly but my testing loss increases.
Also, my training accuracy inscreases while my testing accuracy is especially low and…
wakobu
- 318
- 1
- 11
3
votes
1 answer
tf.gather runs out of bound, while using a custom softmax_loss function, even though it shouldn't
I'm using a small custom function inside of tf.contrib.seq2seq.sequence_loss(softmax_loss_function=[...]) as a custom sofmax_loss_function:
def reduced_softmax_loss(self, labels, logits):
top_logits, indices = tf.nn.top_k(logits,…
JtheB
- 98
- 6
3
votes
1 answer
How to Retrain Sequence to Sequence Neural Network model for Language Translation?
I have trained a seq2seq tensorflow model for translating a sentence from English to Spanish. I trained a model for 615 700 steps, and save the model checkpoints successfully. My training data size for both English and Spanish sentences is 200 000.…
Sandeep
- 369
- 1
- 5
- 16
3
votes
0 answers
how to make decoder in rnn, tensorflow, feeded previous output
I'm wondering how to make decoder in tensorflow rnn, feed it's i th output to (i+1)th input
my inputs have 20 sequence and 3680 dementions
and my outputs have 39 sequence and 3680 dementions
all data is 0~1 number
here is my model
with…
Bonic
- 67
- 5
3
votes
2 answers
Seq2Seq Bidirectional Encoder Decoder in Keras
I am trying to implement a seq2seq encoder-decoder using Keras, with bidirectional lstm on the encoder as follows:
from keras.layers import LSTM,Bidirectional,Input,Concatenate
from keras.models import Model
n_units = 8
n_input = 1
n_output = 1
#…
terenceflow
- 223
- 3
- 9
3
votes
1 answer
Tensorflow seq2seq chatbot always give the same outputs
I'm trying to make a seq2seq chatbot with Tensorflow, but it seems to converge to the same outputs despite different inputs. The model gives different outputs when first initialized, but quickly converges to the same outputs after a few epochs. This…
gundamMC
- 61
- 9
3
votes
1 answer
The shape of the predicted_ids in the outputs of `tf.contrib.seq2seq.BeamSearchDecoder`
What is the shape of the contents in the outputs of tf.contrib.seq2seq.BeamSearchDecoder. I know that it is an instance of class BeamSearchDecoderOutput(scores, predicted_ids, parent_ids), but what is the shape of the scores, predicted_ids and…
lifang
- 1,485
- 3
- 16
- 23
3
votes
1 answer
Undertanding the final_outputs of tf.contrib.seq2seq.dynamic_docode
When I am reading the seq2seq tutorial here, I can't understand the output of function tf.contrib.seq2seq.dynamic_docode.
The tutorial access the output the decoder used in training with sample_id = outputs.sample_idand logits =…
lifang
- 1,485
- 3
- 16
- 23
3
votes
1 answer
Seq2seq LSTM fails to produce sensible summaries
I am training an encoder-decoder LSTM in keras for text summarization and the CNN dataset with the following architecture
Picture of bidirectional encoder-decoder LSTM
I am pretraining the word embedding (of size 256) using skip-gram and
I then…
Moseli Mots'oehli
- 41
- 4
3
votes
1 answer
Add more layers in seq2seq model
In the sample seq2seq code given by fchollet, how can I add more LSTM layers to the encoder and decoder? I'm having some trouble with the shapes and a bit confused in general. Thanks.
S.Mandal
- 171
- 2
- 8
3
votes
1 answer
what is sequence_length field in tf.contrib.seq2seq.TrainingHelper?
Can anyone explain the purpose of sequence length field in tf.contrib.seq2seq.TrainingHelper
Rakesh kumar
- 392
- 5
- 13
3
votes
2 answers
Tensorflow: Troubles with BeamSearchDecoder - dynamic_decode
implementing a seq2seq model with bidirectional multilstm layers with attention and beamsearch.
(Only posted the necessary code to keep it simple)
# helper to create the layers
def make_lstm(rnn_size, keep_prob):
lstm =…
tmmmmmi
- 81
- 3
2
votes
0 answers
How can I execute decoder of ONNX Export from Seq2Seq model
I made an export of the Helsinki model using python optimum and i am trying to run the model with only the onnx environment and implement beam search from scratch because I have to later port this to a system not running python. So I want to…
klsmgföl
- 21
- 3
2
votes
0 answers
Seq2Seq(GNN+RNN) - Odd predictions despite optimized loss
I’d like to ask for your advise/expertise on an issue that I am currently facing.
Summary:
I am training a Seq2Seq model that generates a natural language question based on a graph. Train and validation loss are decreasing throughout training but…
DustyAvocado
- 43
- 4
2
votes
0 answers
What ML model to predict a sequence of numbers from another sequence
I am trying to predict a time series from another time series. The input and the output have the same length but they have a different structure (The input is more noisy), the output has a nice sinusoidal shape. I tried implementing a simple…
Eliott Simon
- 21
- 1