Stack Overflow
Stack Overflow

Questions tagged [lstm]

Long short-term memory. A neural network (NN) architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time. A very popular building block for deep NN.

Long short-term memory neural networks (LSTMs) are a subset of recurrent neural networks. They can take time-series data and make predictions using knowledge of how the system is evolving.

A major benefit to LSTMs is their ability to store and utilize long-term information, not just what they are provided at a particular instance. For more information on LSTMs check out these links from colah's blog post and MachineLearningMastery.

6289 questions
2
votes
1 answer

Truncated Back Propagation (BPTT) for RNN in Tensorflow

https://www.tensorflow.org/tutorials/recurrent#truncated_backpropagation Here, Official TF document says, "In order to make the learning process tractable, it is common practice to create an 'unrolled' version of the network, which contains a…
heyzude
  • 363
  • 1
  • 4
  • 20
2
votes
1 answer

Keras LSTM input features and incorrect dimensional data input

So I'm trying to practice how to use LSTMs in Keras and all parameter (samples, timesteps, features). 3D list is confusing me. So I have some stock data and if the next item in the list is above the threshold of 5 which is +-2.50 it buys OR sells,…
Definity
  • 691
  • 2
  • 11
  • 31
2
votes
1 answer

How to use both Sequential and Non Sequential Features with Tensorflow

My dataset has multiple features containing both sequential data and non sequential data. How can I concatenate or use them together in a model to work it out with all the features rather than just the sequential or the non sequential ones? Thanks…
Daksh M.
  • 4,589
  • 4
  • 30
  • 46
2
votes
0 answers

Using embedding_rnn_seq2seq in tensorflow

The documentation shown in the official documentation seems to be jumping straight into attention models without showing how to use the basic seq2seq model. I'm attempting to translate from a certain date format into one standard method. Some…
sachinruk
  • 9,571
  • 12
  • 55
  • 86
2
votes
1 answer

Using Images as Input to a Bidirectional LSTM Network

I want to know how to process images before being fed as an input to a Bidirectional LSTM. I always get this error whenever I run me code: AttributeError: 'list' object has no attribute 'get_shape' from this line: outputs, _, _ =…
Rocket Pingu
  • 621
  • 9
  • 26
2
votes
1 answer

LSTM for reversing integer sequence

I'm just trying to train a LSTM to reverse a integer sequence. My approach is a modified version of this tutorial, in which he just echoes the input sequence. It goes like this: Generate a random sequence S with length R (possible values range from…
Fernando
  • 7,785
  • 6
  • 49
  • 81
2
votes
1 answer

Where is Keras LSTM Bias added at inference time?

The Keras LSTM implementation outputs kernel weights, recurrent weights and a single bias vector. I would have expected there to be a bias for both the kernel weights and the recurrent weights so I am trying to make sure that I understand where this…
reese0106
  • 2,011
  • 2
  • 16
  • 46
2
votes
2 answers

Keras LSTM dense layer multidimensional input

I'm trying to create a keras LSTM to predict time series. My x_train is shaped like 3000,15,10 (Examples, Timesteps, Features), y_train like 3000,15,1 and I'm trying to build a many to many model (10 input features per sequence make 1 output /…
sbz
  • 100
  • 2
  • 9
2
votes
1 answer

How to train with inputs of variable size?

This question is rather abstract and not necessarily tied to tensorflow or keras. Say that you want to train a language model, and you want to use inputs of different sizes for your LSTMs. Particularly, I'm following this paper:…
Johy
  • 317
  • 1
  • 5
  • 16
2
votes
1 answer

make LSTM cell trainable

Im using the tf.contrib.rnn.MultiRNNCell module to make a multi-layered RNN. I use the following lines to define a 3-layered RNN-LSTM network: n_hidden = 2 num_layers = 3 lstm_cell = tf.contrib.rnn.BasicLSTMCell(n_hidden) stacked_lstm_cell…
python_tensorflow
  • 351
  • 1
  • 3
  • 14
2
votes
1 answer

For Keras LSTM, what is the difference in passing in lag features vs timesteps of features?

I'm getting acquainted with LSTMs and I need clarity on something. I'm modeling a time series using t-300:t-1 to predict t:t+60. My first approach was to set up an LSTM like this: # fake dataset to put words into code: X =…
user4446237
  • 636
  • 8
  • 21
2
votes
0 answers

Keras LSTM time series multi-step predictions has same output for any input

I have a time series of data that has values that represent 'activity' collected by the minute. I set up an LSTM in order to model the data. The LSTM is set to input 300 points, and output the next 60 points. I've tweaked the architecture of the…
user4446237
  • 636
  • 8
  • 21
2
votes
0 answers

LSTM keras package incoherent output

I am mimicking the code from this page, implenting LSTM to predict time series behaviour. I use R so I just translated the content and adapted it to my dataset using R keras package. So, here it goes, I have train_input is a vector of random…
user2478159
  • 133
  • 1
  • 14
2
votes
1 answer

What data rescaling preprocessing do I need to do before training an LSTM NN?

I am trying to build a simple one layered LSTM neural network using keras that has a Dense() output layer with one neuron (used for regression). I know that the training data when I fit() the model have to be rescaled. Many online examples rescale…
mrt
  • 339
  • 1
  • 2
  • 14
2
votes
1 answer

Train a model using lstm and keras

I have an input data like this: x_train = [ [0,0,0,1,-1,-1,1,0,1,0,...,0,1,-1], [-1,0,0,-1,-1,0,1,1,1,...,-1,-1,0] ... [1,0,0,1,1,0,-1,-1,-1,...,-1,-1,0] ] y_train = [1,1,1,0,-1,-1,-1,0,1...,0,1] it is an array of arryas which each…
user8488890
1 2 3
…
99
100