Stack Overflow
Stack Overflow

Questions tagged [sequence-to-sequence]

This tag is used for Google's deprecated seq2seq framework, an encoder-decoder framework for Tensorflow (revamped version is called Neural Machine Translation)

94 questions
4
votes
2 answers

Tensorflow RNN: how to infer a sequence without duplicates?

I'm working on a seq2seq RNN generating an output sequence of labels given a seed label. During the inference step I'd like to generate sequences containing only unique labels (i.e. skip labels that have already been added to the output sequence).…
Tural Gurbanov
  • 742
  • 2
  • 7
  • 27
4
votes
2 answers

In tensorflow, how to calculate sequence loss using output from dynamic_decode

Hi fellow Tensorflowers, I am trying to implement a sequence-to-sequence model using the new seq2seq module that is under development and release with TF1.0 and 1.1. There is a dynamic_decode function that returns logits in the form of a…
mhnatiuk
  • 148
  • 2
  • 11
3
votes
1 answer

Difference between two Sequence to Sequence Models keras (with and without RepeatVector)

I try to understand what the difference between this model describde here, the following one: from keras.layers import Input, LSTM, RepeatVector from keras.models import Model inputs = Input(shape=(timesteps, input_dim)) encoded =…
texaspythonic
  • 63
  • 7
3
votes
1 answer

What does the "source hidden state" refer to in the Attention Mechanism?

The attention weights are computed as: I want to know what the h_s refers to. In the tensorflow code, the encoder RNN returns a tuple: encoder_outputs, encoder_state = tf.nn.dynamic_rnn(...) As I think, the h_s should be the encoder_state, but the…
imhuay
  • 271
  • 1
  • 2
  • 11
3
votes
1 answer

Is tensorflow embedding_lookup differentiable?

Some of the tutorials I came across, described using a randomly initialized embedding matrix and then using the tf.nn.embedding_lookup function to obtain the embeddings for the integer sequences. I am under the impression that since the…
Animesh Karnewar
  • 416
  • 1
  • 8
  • 17
3
votes
1 answer

How to modify the Tensorflow Sequence2Sequence model to implement Bidirectional LSTM rather than Unidirectional one?

Refer to this post to know the background of the problem: Does the TensorFlow embedding_attention_seq2seq method implement a bidirectional RNN Encoder by default? I am working on the same model, and want to replace the unidirectional LSTM layer with…
Leena Shekhar
  • 31
  • 3
2
votes
1 answer

Loss function negative log likelihood giving loss despite perfect accuracy

I am debugging a sequence-to-sequence model and purposely tried to perfectly overfit a small dataset of ~200 samples (sentence pairs of length between 5-50). I am using negative log-likelihood loss in pytorch. I get low loss (~1e^-5), but the…
headache666
  • 21
  • 2
2
votes
0 answers

Keras Looping LSTM layers

I am trying to build a model which is basically sequence to sequence model but i have a special encoder namely "Secondary Encoder". Timesteps in Secondary Encoder = 300 this encoder has a special property, in essence it is a GRU, but at each…
Harsh
  • 21
  • 2
2
votes
0 answers

How to take last hidden state of bidirectional LSTM in Pytorch?

I'm not sure how to select the last hidden/cell states in a bidirectional LSTM in Pytorch. output, (hn, cn) = bi_lstm(input, (h0, c0)) How can I use output, hn and cn in order to extract the last forward and backward hidden states? In the…
miditower
  • 107
  • 2
  • 9
2
votes
2 answers

keras pad_sequence for string data type

I have a list of sentences. I want to add padding to them; but when I use keras pad_sequence like this: from keras.preprocessing.sequence import pad_sequences s = [["this", "is", "a", "book"], ["this", "is", "not"]] g = pad_sequences(s, dtype='str',…
Behnaz Moradabadi
  • 21
  • 2
2
votes
1 answer

Creating a custom metric in Keras for sequence to sequence learning

I want to write a custom metric in Keras (python) to evaluate the performance of my sequence to sequence model as I train. Sequences are one-hot encoded and the tokens are words instead of characters. I want it to report the number of sequences that…
Hannah Morgan
  • 133
  • 1
  • 9
2
votes
0 answers

I do not know why in my Keras neural network model, the prediction shape is not consistent with the shape of labels while training?

I have built a Keras ConvLSTM neural network, and I want to predict one frame ahead based on a sequence of 10 time steps: Model: from keras.models import Sequential from keras.layers.convolutional import Conv3D from…
MRM
  • 1,099
  • 2
  • 12
  • 29
2
votes
0 answers

I would like to have an example of using Tensorflow ConvLSTMCell

I would like to have a small example of building an encoder-decoder network using Tensorflow ConvLSTMCell. Thanks
MRM
  • 1,099
  • 2
  • 12
  • 29
2
votes
1 answer

TensorFlow sequence_loss with label_smoothing

Would it be possible to use the label_smoothing feature from tf.losses.softmax_cross_entropy with tf.contrib.seq2seq.sequence_loss ? I can see that sequence_loss optionally takes a softmax_loss_function as parameter. However, this function would…
George
  • 934
  • 8
  • 23
2
votes
1 answer

Tensorflow seq2seq: Tensor' object is not iterable

I am using seq2seq below code, I found below error: cell = tf.nn.rnn_cell.BasicLSTMCell(size) a, b = tf.nn.dynamic_rnn(cell, seq_input, dtype=tf.float32) cell_a = tf.contrib.rnn.OutputProjectionWrapper(cell, frame_dim) dec_output=…
Mike Flanagan
  • 23
  • 3
1
2
3 4 5 6 7