8

I have an LSTM in Keras that I am training to predict on time series data. I want the network to output predictions on each timestep, as it will receive a new input every 15 seconds. So what I am struggling with is the proper way to train it so that it will output h_0, h_1, ..., h_t, as a constant stream as it receives x_0, x_1, ...., x_t as a stream of inputs. Is there a best practice for doing this?

enter image description here

Rob
  • 3,333
  • 5
  • 28
  • 71

1 Answers1

5

You can enable statefulness in your LSTM layers by setting stateful=True. This changes the behavior of the layer to always use the state of the previous invocation of the layer instead of resetting it for each layer.call(x).

For example an LSTM layer with 32 units with batch size 1, sequence length 64 and feature length 10:

LSTM(32, stateful=True, batch_input_shape=(1,64,10))

With this successive calls of predict will use the previous states.

nemo
  • 55,207
  • 13
  • 135
  • 135
  • 1
    And so .reset_states() would be the function to start a new sequence of inputs? Very cool, thank you! – Rob Jun 29 '16 at 14:04
  • Yep. Forgot to mention that, sorry. – nemo Jun 29 '16 at 15:05
  • @nemo I have a followup question [here](http://stackoverflow.com/questions/38313673/lstm-with-keras-for-mini-batch-training-and-online-testing). Would you mind taking a look? – BoltzmannBrain Jul 12 '16 at 16:55