15

I have sequential data and I declared a LSTM model which predicts y with x in Keras. So if I call model.predict(x1) and model.predict(x2), Is it correct to call model.reset_states between the two predict() explicitly? Does model.reset_states clear history of inputs, not weights, right?

# data1
x1 = [2,4,2,1,4]
y1 = [1,2,3,2,1]

# dat2
x2 = [5,3,2,4,5]
y2 = [5,3,2,3,2]

And in my actual code, I use model.evaluate(). In evaluate(), is reset_states called implicitly for each data sample?

model.evaluate(dataX, dataY)
prosti
  • 42,291
  • 14
  • 186
  • 151
jef
  • 3,890
  • 10
  • 42
  • 76

2 Answers2

26

reset_states clears only the hidden states of your network. It's worth to mention that depending on if the option stateful=True was set in your network - the behaviour of this function might be different. If it's not set - all states are automatically reset after every batch computations in your network (so e.g. after calling fit, predict and evaluate also). If not - you should call reset_states every time, when you want to make consecutive model calls independent.

Marcin Możejko
  • 39,542
  • 10
  • 109
  • 120
  • I got it. If I do not set stateful option (so default=false), I do not need to call reset_states, right? And could you tell me what kind of cases I should use stateful=True? – jef Mar 14 '17 at 00:49
  • 3
    Yes, you are right. `stateful=True` is usually used when you want to treat consecutive batches as consequtive inputs. In this case model is treating consequtive batches the same as it were in the same batch. – Marcin Możejko Mar 14 '17 at 19:58
  • 2
    @MarcinMożejko How does the model learn when the hidden states are cleared after every batch training? What does `clear` mean here? – asn Jan 16 '19 at 17:47
4

If you use explicitly either of:

model.reset_states()

to reset the states of all layers in the model, or

layer.reset_states()

to reset the states of a specific stateful RNN layer (also LSTM layer), implemented here:

def reset_states(self, states=None):
  if not self.stateful:
     raise AttributeError('Layer must be stateful.')

this means your layer(s) must be stateful.

In LSTM you need to:

  • explicitly specify the batch size you are using, by passing a batch_size argument to the first layer in your model or batch_input_shape argument

  • set stateful=True.

  • specify shuffle=False when calling fit().

The benefits of using stateful models are probable best explained here.

prosti
  • 42,291
  • 14
  • 186
  • 151