Questions tagged [sequence-modeling]
7 questions
3
votes
2 answers
How can I do a sequence-to-sequence model (RNN / LSTM) with Keras with fixed length data?
What I'm trying to do seems so simple, but I can't find any examples online. First, I'm not working in language, so all of the embedding stuff adds needless complexity to my task.
I have input, in the form of a (1, 1000) vector. They are time-series…
Shamoon
- 41,293
- 91
- 306
- 570
2
votes
1 answer
Transformer based decoding
Can the decoder in a transformer model be parallelized like the encoder? As far as I understand the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder this is not possible (in both training and…
shiredude95
- 560
- 3
- 7
1
vote
0 answers
How to implement Bi-Directional Conv LSTM in Pytorch
import torch
from torch import nn
def initialize_weights(self, layer):
"""Initialize a layer's weights and biases.
Args:
layer: A PyTorch Module's layer."""
if isinstance(layer, (nn.BatchNorm2d,…
Rituraj Dutta
- 55
- 5
0
votes
0 answers
TF-IDF vector vs a vector of tokens
Some background:
I have data structured as the a TFIDF vector of shape (15637, 31635) and this is the input this vector into the LSTM layer. The longest word in my vocabulary is 305 words and each TFIDF vector has length 31635 because the total…
Utpal Mattoo
- 890
- 3
- 17
- 41
0
votes
1 answer
How the function nn.LSTM behaves within the batches/ seq_len?
I’m currently learning to use nn.LSTM with pytorch and had to ask how the function is working.
Basically I’m trying to feed my dataset matrix (M x N).
Since the dataset is a matrix, I wanted to feed the dataset recursively(as timesteps) into the…
jinujanu
- 13
- 3
0
votes
1 answer
how to create train - dev - test sets from a given dataset in sequence models
Assume that we have the following dataset, where 's' stands for 'step'.
f1 f2 f3 f4 target
1 2 3 4 5
2 3 4 5 6
3 4 5 6 7
4 5 6 7 8
5 6 7 8 9
The model consists of 4 (time) steps. And it gives a…
edn
- 1,981
- 3
- 26
- 56
0
votes
1 answer
Is there any other reason why we make sequence length the same using padding?
Is there any other reason why we make sequence length the same length using padding? Other than in order to do matrix multiplication (therefore doing parallel computation).
aerin
- 20,607
- 28
- 102
- 140