Stack Overflow
Stack Overflow

Questions tagged [lstm]

Long short-term memory. A neural network (NN) architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time. A very popular building block for deep NN.

Long short-term memory neural networks (LSTMs) are a subset of recurrent neural networks. They can take time-series data and make predictions using knowledge of how the system is evolving.

A major benefit to LSTMs is their ability to store and utilize long-term information, not just what they are provided at a particular instance. For more information on LSTMs check out these links from colah's blog post and MachineLearningMastery.

6289 questions
16
votes
1 answer

Keras attention layer over LSTM

I'm using keras 1.0.1 I'm trying to add an attention layer on top of an LSTM. This is what I have so far, but it doesn't work. input_ = Input(shape=(input_length, input_dim)) lstm = GRU(self.HID_DIM, input_dim=input_dim, input_length = input_length,…
siamii
  • 23,374
  • 28
  • 93
  • 143
16
votes
1 answer

Input to LSTM network tensorflow

I have a time series of length t (x0, ...,xt) each of the xi is a d-dimension vector i.e. xi=(x0i, x1i, ...., xdi). Thus my input X is of shape [batch_size, d] The input for the tensorflow LSTM should be of size [batchSize, hidden_size]. My…
ofer-a
  • 521
  • 5
  • 21
15
votes
3 answers

Multivariate input LSTM in pytorch

I would like to implement LSTM for multivariate input in Pytorch. Following this article https://machinelearningmastery.com/how-to-develop-lstm-models-for-time-series-forecasting/ which uses keras, the input data are in shape of (number of samples,…
Tomas Trdla
  • 1,142
  • 1
  • 11
  • 24
15
votes
1 answer

How to prepare data for LSTM when using multiple time series of different lengths and multiple features?

I have a dataset from a number of users (nUsers). Each user is sampled randomly in time (non-constant nSamples for each user). Each sample has a number of features (nFeatures). For example: nUsers = 3 ---> 3 users nSamples = [32, 52, 21] ---> first…
AR_
  • 468
  • 6
  • 18
15
votes
1 answer

Understanding LSTM model using tensorflow for sentiment analysis

I am trying to learn LSTM model for sentiment analysis using Tensorflow, I have gone through the LSTM model. Following code (create_sentiment_featuresets.py) generates the lexicon from 5000 positive sentences and 5000 negative sentences. import…
LinuxBeginner
  • 185
  • 8
15
votes
3 answers

Padding time-series subsequences for LSTM-RNN training

I have a dataset of time series that I use as input to an LSTM-RNN for action anticipation. The time series comprises a time of 5 seconds at 30 fps (i.e. 150 data points), and the data represents the position/movement of facial features. I sample…
Michele Tonutti
  • 4,298
  • 1
  • 21
  • 22
15
votes
2 answers

How to use model.reset_states() in Keras?

I have sequential data and I declared a LSTM model which predicts y with x in Keras. So if I call model.predict(x1) and model.predict(x2), Is it correct to call model.reset_states between the two predict() explicitly? Does model.reset_states clear…
jef
  • 3,890
  • 10
  • 42
  • 76
15
votes
1 answer

LSTM with Keras for mini-batch training and online testing

I would like to implement an LSTM in Keras for streaming time-series prediction -- i.e., running online, getting one data point at a time. This is explained well here, but as one would assume, the training time for an online LSTM can be…
BoltzmannBrain
  • 5,082
  • 11
  • 46
  • 79
15
votes
2 answers

LSTM module for Caffe

Does anyone know if there exists a nice LSTM module for Caffe? I found one from a github account by russel91 but apparantly the webpage containing examples and explanations disappeared (Formerly http://apollo.deepmatter.io/ --> it now redirects only…
mcExchange
  • 6,154
  • 12
  • 57
  • 103
14
votes
2 answers

Dropout layer before or after LSTM. What is the difference?

Suppose that we have an LSTM model for time series forecasting. Also, this is a multivariate case, so we're using more than one feature for training the model. ipt = Input(shape = (shape[0], shape[1]) x = Dropout(0.3)(ipt) ## Dropout before…
Eghbal
  • 3,892
  • 13
  • 51
  • 112
14
votes
1 answer

How visualize attention LSTM using keras-self-attention package?

I'm using (keras-self-attention) to implement attention LSTM in KERAS. How can I visualize the attention part after training the model? This is a time series forecasting case. from keras.models import Sequential from keras_self_attention import…
Eghbal
  • 3,892
  • 13
  • 51
  • 112
14
votes
1 answer

Why does my keras LSTM model get stuck in an infinite loop?

I am trying to build a small LSTM that can learn to write code (even if it's garbage code) by training it on existing Python code. I have concatenated a few thousand lines of code together in one file across several hundred files, with each file…
Shamoon
  • 41,293
  • 91
  • 306
  • 570
14
votes
3 answers

Error when checking target: expected dense_1 to have 3 dimensions, but got array with shape (118, 1)

I'm training a model to predict the stock price and input data is close price. I use 45 days data to predict the 46th day's close price and a economic Indicator to be second feature, here is the model: model = Sequential() model.add( LSTM( 512,…
Chris Wong
  • 143
  • 1
  • 1
  • 6
14
votes
2 answers

Multilayer Seq2Seq model with LSTM in Keras

I was making a seq2seq model in keras. I had built single layer encoder and decoder and they were working fine. But now I want to extend it to multi layer encoder and decoder. I am building it using Keras Functional API. Training:- Code for…
SAGAR
  • 151
  • 1
  • 7
14
votes
3 answers

Keras LSTM: a time-series multi-step multi-features forecasting - poor results

I have a time series dataset containing data from a whole year (date is the index). The data was measured every 15 min (during whole year) which results in 96 timesteps a day. The data is already normalized. The variables are correlated. All the…
Bolesław Maliszewski
  • 163
  • 1
  • 1
  • 7
1 2 3
…
99 100