Stack Overflow
Stack Overflow

Questions tagged [sequence-to-sequence]

This tag is used for Google's deprecated seq2seq framework, an encoder-decoder framework for Tensorflow (revamped version is called Neural Machine Translation)

94 questions
1
vote
2 answers

How do I convert the output data from the decoder LSTM back to words?

I have an encoder decoder network mimicking the one produced in this tutorial: https://towardsdatascience.com/how-to-implement-seq2seq-lstm-model-in-keras-shortcutnlp-6f355f3e5639 However the output of the decoder LSTM will be numbers between 0 and…
Haztec
  • 31
  • 3
1
vote
1 answer

Embedding layer in neural machine translation with attention

I am trying to understanding how to implement a seq-to-seq model with attention from this website. My question: Is nn.embedding just returns some IDs for each word, so the embedding for each word would be the same during whole training? Or are they…
Kadaj13
  • 1,423
  • 3
  • 17
  • 41
1
vote
1 answer

Exception in Sequence-to-Sequence model in Keras

I am trying to build a sequence to sequence model in Keras using LSTM and dense neural network. The encoder encodes the input, the encoded state and the inputs are then concatenated and fed into a decoder which is an lstm + dense neural network…
Abdul Rahman
  • 1,294
  • 22
  • 41
1
vote
0 answers

What is num_units argument in BahdanauAttention?

I don't understand the argument num_units in TensorFlow's BahdanauAttention. It's defined as: num_units: The depth of the query mechanism. which is unclear to me.
edoost
  • 101
  • 1
  • 8
1
vote
1 answer

Merging sequence embedding with Time Series Features

I am having trouble around certain aspects of the Keras implementation of LSTM. This is a description of my problem: I am trying to train a model for word correctness prediction. My model has two types of inputs: A word sequence (sentence) And a…
user3487059
  • 21
  • 4
1
vote
1 answer

How to build a decoder using dynamic rnn in Tensorflow?

I know how to build an encoder using dynamic rnn in Tensorflow, but my question is how can we use it for decoder? Because in decoder at each time step we should feed the prediction of previous time step. Thanks in advance!
MRM
  • 1,099
  • 2
  • 12
  • 29
1
vote
0 answers

TensorFlow BeamSearchDecoder outputs the sample_id as (actual sample_id+1)

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes. Based on the NMT tutorial, I am writing a customized code for my own task. OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Linux Ubuntu…
Seonghyeon Park
  • 56
  • 3
1
vote
1 answer

What is the difference between ensembling and averaging models?

In machine translation, sequence-to-sequence models have become very popular. They often use a few tricks to improve performance, such as ensembling or averaging a set of models. The logic here is that the errors will then "average out". As I…
RNRug
  • 55
  • 5
1
vote
0 answers

No op named GatherTree when using BeamSearchDecoder

I'm implementing a Seq2Seq model with TensorFlow. My code works using the Greedy Decoder, but when I was using BeamSearchDecoder to improve the performance, I encountered this error: Traceback (most recent call last): File…
Michael Chen
  • 11
  • 3
1
vote
2 answers

Using CNTK to generate sequence by sampling at each generation step

In a seq2seq model with an encoder and a decoder, at each generation step a softmax layer outputs a distribution over the entire vocabulary. In CNTK, a greedy decoder can be implemented easily by using the C.hardmax function. It looks like this. def…
meijiesky
  • 11
  • 2
1
vote
0 answers

Create input/output sequences from separate columns in csv for seq2seq decoder in tensorflow

I am trying to experiment with tensorflow seq2seq and I am having trouble coming up with a good way to add the "GO", "EOS" + "PAD" elements a sequence of labels. I am reading this data from .csv using tf.TextLineReader and the .csv I have created…
reese0106
  • 2,011
  • 2
  • 16
  • 46
1
vote
1 answer

Tensor flow continuous text sequence-to-sequence. Why batch?

I'm working through building a sequence-to-sequence shakespeare predictor and looking at sample code it seems to do batching in groups of 50 characters. I'm a little confused by this. If the text is continuous and you are processing in…
raeldor
  • 503
  • 3
  • 11
1
vote
0 answers

Batching Seq2Seq model with Tensorflow Experiment

I'm trying to convert a Seq2Seq model I've run locally to use Tensorflow distrubution capabilities using Estimator and Experiment. The basic feature and targets are set up as follows: for every input and response (translation or prompt and…
Eric H.
  • 6,894
  • 8
  • 43
  • 62
1
vote
0 answers

Tensorflow 1.0 Seq2Seq Decoder function

I'm trying to make a Seq2Seq Regression example for time-series analysis and I've used the Seq2Seq library as presented at the Dev Summit, which is currently the code on the Tensorflow GitHub branch r1.0. I have difficulties understanding how the…
Vithursan Thangarasa
  • 11
  • 1
1
vote
0 answers

Sampled softmax loss over variable sequence batches?

Background info: I'm working on sequence-to-sequence models, and right now my model accepts variable-length input tensors (not lists) with input shapes corresponding to [batch size, sequence length]. However, in my implementation, sequence length is…
Brandon McKinzie
  • 46
  • 1
  • 5
1 2 3
4
5 6 7