Stack Overflow
Stack Overflow

Questions tagged [seq2seq]

Seq2Seq is a sequence to sequence learning add-on for the python deep learning library.

318 questions
2
votes
1 answer

Tensorflow keras Bidirectional LSTM for text summarization

I am trying to implement a bidirectional LSTM for text summarization. I have issue with the inference section. The dimension does not match. This is my model: latent_dim = 300 embedding_dim=100 # Encoder encoder_inputs =…
tsann
  • 89
  • 4
2
votes
1 answer

How much is the dimension of some bidirectional LSTM layers?

I read a paper about machine translation, and it uses projection layer. Its encoder has 6 bidirectional LSTM layers. If input embedding dimension is 512, how much will be the dimension of the encoder output? 512*2**5? The paper's link:…
kintsuba
  • 139
  • 2
  • 7
2
votes
1 answer

Keras LSTM for converting sentences to document context vector

I read the following blog post and tried to implement it via Keras: https://andriymulyar.com/blog/bert-document-classification Now, Im quite new to Keras and I do not understand how to use "seq2seq neural networks" to condens a sequence of subchunks…
Felix
  • 313
  • 1
  • 3
  • 22
2
votes
1 answer

how to build a custom bidirectional encoder for seq2seq with tf2?

class Encoder(tf.keras.Model): def __init__(self, vocab_size, embedding_dim, enc_units, batch_sz): super(Encoder, self).__init__() self.batch_sz = batch_sz self.enc_units = enc_units self.embedding =…
wyh127
  • 23
  • 3
2
votes
0 answers

Max Sequence length in Seq2Seq - Attention is all you need

I have gone through the paper Attention is all you need and though I think I understood the overall idea behind what is happening, I am pretty confused with the way the input is being processed. Here are my doubts, and for simplicity, let's assume…
Kakarot
  • 175
  • 1
  • 3
  • 10
2
votes
0 answers

How to train a simple, vanilla transformers translation model from scratch with Fairseq

I have been familiarizing myself with the fairseq library recently, and have tried a couple of pretrained models. I thought that a good way to teach myself would be to train a plain vanilla transformers model with the data I have, and then I can…
md1630
  • 841
  • 1
  • 10
  • 28
2
votes
0 answers

LSTM many-to-many synced sequence input and output Pytorch loss not decreasing?

I am working on a a project where I built an LSTM model for seq2seq, where I have a synced sequence input and output. My audio time series is 32000 in length and my labels are also 32000 in length. And we wish to make a classification (fake or real…
javacash
  • 41
  • 3
2
votes
0 answers

Why does my Seq2Seq model only predicts token?

I'm creating a video captioning seq2seq model. My encoder inputs are video features, and my decoder inputs are captions, beggining with a token and padded with tokens. Problem: During the teacher forcing training period, after few iterations, it…
wakobu
  • 318
  • 1
  • 11
2
votes
0 answers

How to generate sequence correctly with encoder-decoder lstm?

I am implementing some code to generate labeled data for Natural Language Understanding (NLU) from the article "Labeled Data Generation with Encoder-decoder LSTM for Semantic Slot Filling"…
Daniel Martin Arteaga Melendez
  • 21
  • 1
2
votes
1 answer

Transformer based decoding

Can the decoder in a transformer model be parallelized like the encoder? As far as I understand the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder this is not possible (in both training and…
shiredude95
  • 560
  • 3
  • 7
2
votes
2 answers

Mistake in pytorch attention seq2seq tutorial?

I am writing a sequence to sequence neural network in Pytorch. In the official Pytorch seq2seq tutorial, there is code for an Attention Decoder that I cannot understand/think might contain a mistake. It computes the attention weights at each time…
ludog
  • 74
  • 1
  • 8
2
votes
1 answer

Provide Tensorflow Seq2Seq output as input at next step (inference)

I would like to create a Seq2Seq model to forecast time series data. I am using the InferenceHelper and I am struggling with the sample_fn parameter. I would like to pass the decoder output of each cell through a dense layer in order to generate a…
peterk
  • 21
  • 2
2
votes
0 answers

Error model/att_seq2seq/Minimum:0 is both fed and fetched

I have successfully exported the seq2seq model in SavedModel format with the following code source_tokens_ph = tf.placeholder(dtype=tf.string, shape=(1, None)) source_len_ph = tf.placeholder(dtype=tf.int32, shape=(1,)) features_serve = { …
Sathyamoorthy R
  • 383
  • 4
  • 19
2
votes
1 answer

What's the point of training the seq2seq model if it is not used in inference?

In official Keras seq2seq example (I'll include it at the bottom) they train the model with the fit function, but they don't even use that model anywhere in the decoding process to test the model on new data. I am trying to train a seq2seq model…
Carson P
  • 313
  • 3
  • 13
2
votes
2 answers

Keras seq2seq stacked layers

In tutorial: https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html we have one layer seq2seq model. I would like to extend this model with one additional layer on encoder side and one additional layer on…
Amel Music
  • 119
  • 2
  • 9
1 2 3
…
21 22