Stack Overflow
Stack Overflow

Questions tagged [lstm]

Long short-term memory. A neural network (NN) architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time. A very popular building block for deep NN.

Long short-term memory neural networks (LSTMs) are a subset of recurrent neural networks. They can take time-series data and make predictions using knowledge of how the system is evolving.

A major benefit to LSTMs is their ability to store and utilize long-term information, not just what they are provided at a particular instance. For more information on LSTMs check out these links from colah's blog post and MachineLearningMastery.

6289 questions
43
votes
1 answer

How to correctly give inputs to Embedding, LSTM and Linear layers in PyTorch?

I need some clarity on how to correctly prepare inputs for batch-training using different components of the torch.nn module. Specifically, I'm looking to create an encoder-decoder network for a seq2seq model. Suppose I have a module with these three…
Silpara
  • 639
  • 1
  • 8
  • 14
41
votes
1 answer

Shuffling training data with LSTM RNN

Since an LSTM RNN uses previous events to predict current sequences, why do we shuffle the training data? Don't we lose the temporal ordering of the training data? How is it still effective at making predictions after being trained on shuffled…
hellowill89
  • 1,538
  • 2
  • 15
  • 26
41
votes
2 answers

Neural Network LSTM input shape from dataframe

I am trying to implement an LSTM with Keras. I know that LSTM's in Keras require a 3D tensor with shape (nb_samples, timesteps, input_dim) as an input. However, I am not entirely sure how the input should look like in my case, as I have just one…
dreamer
  • 1,192
  • 5
  • 20
  • 42
40
votes
4 answers

ValueError: Input 0 is incompatible with layer lstm_13: expected ndim=3, found ndim=4

I am trying for multi-class classification and here are the details of my training input and output: train_input.shape= (1, 95000, 360) (95000 length input array with each element being an array of 360 length) train_output.shape = (1, 95000, 22)…
Urja Pawar
  • 1,087
  • 1
  • 15
  • 29
39
votes
4 answers

How do I mask a loss function in Keras with the TensorFlow backend?

I am trying to implement a sequence-to-sequence task using LSTM by Keras with the TensorFlow backend. The inputs are English sentences with variable lengths. To construct a dataset with 2-D shape [batch_number, max_sentence_length], I add EOF at the…
Shuaaai
  • 403
  • 1
  • 4
  • 8
38
votes
1 answer

Understanding Keras LSTMs: Role of Batch-size and Statefulness

Sources There are several sources out there explaining stateful / stateless LSTMs and the role of batch_size which I've read already. I'll refer to them later in my post: [1]…
ascripter
  • 5,665
  • 12
  • 45
  • 68
37
votes
1 answer

Multivariate LSTM with missing values

I am working on a Time Series Forecasting problem using LSTM. The input contains several features, so I am using a Multivariate LSTM. The problem is that there are some missing values, for example: Feature 1 Feature 2 ... Feature n 1 …
Marco
  • 1,195
  • 3
  • 18
  • 30
37
votes
7 answers

Why does Keras LSTM batch size used for prediction have to be the same as fitting batch size?

When using a Keras LSTM to predict on time series data I've been getting errors when I'm trying to train the model using a batch size of 50, while then trying to predict on the same model using a batch size of 1 (ie just predicting the next value). …
DanielSon
  • 1,415
  • 4
  • 27
  • 40
36
votes
3 answers

Error when checking model input: expected lstm_1_input to have 3 dimensions, but got array with shape (339732, 29)

My input is simply a csv file with 339732 rows and two columns : the first being 29 feature values, i.e. X the second being a binary label value, i.e. Y I am trying to train my data on a stacked LSTM model: data_dim = 29 timesteps = 8 num_classes…
Saurav--
  • 1,530
  • 2
  • 15
  • 33
35
votes
3 answers

Understanding a simple LSTM pytorch

import torch,ipdb import torch.autograd as autograd import torch.nn as nn import torch.nn.functional as F import torch.optim as optim from torch.autograd import Variable rnn = nn.LSTM(input_size=10, hidden_size=20, num_layers=2) input =…
Abhishek Bhatia
  • 9,404
  • 26
  • 87
  • 142
31
votes
2 answers

Pytorch LSTM vs LSTMCell

What is the difference between LSTM and LSTMCell in Pytorch (currently version 1.1)? It seems that LSTMCell is a special case of LSTM (i.e. with only one layer, unidirectional, no dropout). Then, what's the purpose of having both implementations?…
dkv
  • 6,602
  • 10
  • 34
  • 54
30
votes
6 answers

tensorflow warning - Found untraced functions such as lstm_cell_6_layer_call_and_return_conditional_losses

I'm using tensorflow2.4, and new to tensorflow Here's the code model = Sequential() model.add(LSTM(32, input_shape=(X_train.shape[1:]))) model.add(Dropout(0.2)) model.add(Dense(1, activation='linear')) model.compile(optimizer='rmsprop',…
Cherry Wu
  • 3,844
  • 9
  • 43
  • 63
30
votes
2 answers

How to get the output shape of a layer in Keras?

I have the following code in Keras (Basically I am modifying this code for my use) and I get this error: 'ValueError: Error when checking target: expected conv3d_3 to have 5 dimensions, but got array with shape (10, 4096)' Code: from keras.models…
MRM
  • 1,099
  • 2
  • 12
  • 29
29
votes
3 answers

PyTorch - How to deactivate dropout in evaluation mode

This is the model I defined it is a simple lstm with 2 fully connect layers. import copy import torch import torch.nn as nn import torch.nn.functional as F import torch.optim as optim class mylstm(nn.Module): def __init__(self,input_dim,…
Tommy Yu
  • 1,080
  • 3
  • 11
  • 30
29
votes
2 answers

Building a mutlivariate, multi-task LSTM with Keras

Preamble I am currently working on a Machine Learning problem where we are tasked with using past data on product sales in order to predict sales volumes going forward (so that shops can better plan their stocks). We essentially have time series…
Karl
  • 5,573
  • 8
  • 50
  • 73
1
2
3
…
99 100