Stack Overflow
Stack Overflow

Questions tagged [seq2seq]

Seq2Seq is a sequence to sequence learning add-on for the python deep learning library.

318 questions
4
votes
1 answer

RuntimeError: The size of tensor a (1024) must match the size of tensor b (512) at non-singleton dimension 3

I am doing the following operation, energy.masked_fill(mask == 0, float("-1e20")) my python traces are below, File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input,…
vinsent paramanantham
  • 953
  • 3
  • 15
  • 34
4
votes
2 answers

OSError: [E050] Can't find model 'de'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory

So I am trying to make a seq to seq model for translating german to english using pytorch on online notebook like kaggle notebook and google colab import torch import torch.nn as nn import torch.optim as optim from torchtext.datasets import…
simarpreetsingh.019
  • 129
  • 3
  • 13
4
votes
1 answer

How to use tensorflow Attention layer?

I am trying to understand how to use the tf.keras.layers.Attention shown here: Tensorflow Attention Layer I am trying to use it with encoder decoder seq2seq model. Below is my code: encoder_inputs = Input(shape=(max_len_text,)) enc_emb =…
Jasmine Padhye
  • 45
  • 5
4
votes
2 answers

Specifying a seq2seq autoencoder. What does RepeatVector do? And what is the effect of batch learning on predicting output?

I am building a basic seq2seq autoencoder, but I'm not sure if I'm doing it correctly. model = Sequential() # Encoder model.add(LSTM(32, activation='relu', input_shape =(timesteps, n_features ), return_sequences=True)) model.add(LSTM(16,…
Garry
  • 179
  • 13
4
votes
0 answers

Seq2Seq Model (DL4J) Making Absurd Predictions

I am trying to implement a Seq2Seq Predictor Model in DL4J. What I ultimately want is to use a time series of INPUT_SIZE data points to predict the following time series of OUTPUT_SIZE data points using this type of model. Each data point has…
Alerra
  • 1,479
  • 11
  • 25
4
votes
0 answers

Tensorflow Sequence to Sequence CustomHelper

There are limited amount of documentation on Sequence to Sequence CustomHelper helper = tf.contrib.seq2seq.CustomHelper(initialize_fn = initialize_fn,sample_fn = sample_fn, next_inputs_fn = next_inputs_fn) in Tensorflow. Would anyone explain the…
Pierre Hoshyar
  • 73
  • 6
4
votes
1 answer

using LSTMs Decoder without teacher forcing - Tensorflow

I'm trying to build a sequence to sequence model in Tensorflow , I have followed several tutorials and all is good. Untill I reached a point where I decided to remove the teacher forcing in my model . below is a sample of decoder network that I'm…
mousa alsulaimi
  • 316
  • 1
  • 14
4
votes
1 answer

Preprocessing for seq2seq model

I am trying to build a seq2seq model , I tried to follow Tensorflow official tutorial but there is no preprocessing steps mentioned. I tried to search on web , every tutorial start from model , There is no preprocessing steps info. I need some info…
Aaditya Ura
  • 12,007
  • 7
  • 50
  • 88
3
votes
1 answer

What's the difference between LSTM and Seq2Seq (M to 1)

I want to ask the LSTM can be modeled as many-to-one. However, Seq2Seq can also be modeled as many-to-one. (M to N, when N is one). So, what is the difference?
zhe zheng
  • 89
  • 1
  • 4
3
votes
0 answers

How to use pre-trained FastText embeddings with existing Seq2Seq model?

I'm new in NLP and I am trying to understand how to use pre-trained word embeddings like fastText with the existing Seq2Seq model. The Seq2Seq model I'm working with is the following. The encoder is simple and the decoder is Pointer Generator…
sarah
  • 31
  • 1
3
votes
1 answer

Training seq2seq model on Google Colab TPU with big dataset - Keras

I'm trying to train a sequence to sequence model for machine translation using Keras on Google Colab TPU. I have a dataset which I can load in memory but I have to preprocess to it to feed it to the model. In particular I need to convert the target…
Marco Ripamonti
  • 267
  • 1
  • 3
  • 12
3
votes
1 answer

How conv1d pytorch operates on a sequence of characters or frames?

I understand convolution filters when applied to an image (e.g. an 224x224 image with 3 in-channels transformed by 56 total filters of 5x5 conv to a 224x224 image with 56 out-channels). The key is that there are 56 different filters each with 5x5x3…
Joe Black
  • 625
  • 6
  • 19
3
votes
1 answer

Pytorch inconsistent size with pad_packed_sequence, seq2seq

I'm having some inconsistencies with the output of a encoder I got from this github . The encoder looks as follows: class Encoder(nn.Module): r"""Applies a multi-layer LSTM to an variable length input sequence. """ def __init__(self,…
eljiwo
  • 687
  • 1
  • 8
  • 29
3
votes
1 answer

InvalidArgumentError on Decoder Model during Inference, for LSTM-based Seq2Seq on Tensorflow 2.0

versions: Python 3.6.9, Tensorflow 2.0.0, CUDA 10.0, CUDNN 7.6.1, Nvidia driver version 410.78. I'm trying to port a LSTM-based Seq2Seq tf.keras model to tensorflow 2.0 Right now I'm facing the following error when I try to call predict on the…
Felipe
  • 11,557
  • 7
  • 56
  • 103
3
votes
1 answer

TimeDistributed(Dense) vs Dense in seq2seq

Given the code below encoder_inputs = Input(shape=(16, 70)) encoder = LSTM(latent_dim, return_state=True) encoder_outputs, state_h, state_c = encoder(encoder_inputs) # We discard `encoder_outputs` and only keep the states. encoder_states = [state_h,…
william007
  • 17,375
  • 25
  • 118
  • 194
1
2
3
…
21 22