2

I am trying to build a simple one layered LSTM neural network using keras that has a Dense() output layer with one neuron (used for regression).

I know that the training data when I fit() the model have to be rescaled. Many online examples rescale both the input and the output within the range [0,1], others within the range [-1,1].

What is the right choice for rescaling the input and the output for a simple regression problem like mine and why? I know it has to do with the activation functions used e.g. sigmoid outputs values within [0,1].

I have seen that you can select the activation function and recurrent activation. How are the activation functions implemented in the context of an LSTM() in keras (e.g. the input gate has a 'sigmoid')?

Thanks!

mrt
  • 339
  • 1
  • 2
  • 14
  • you would probably get an answer to this question on [this site](https://stats.stackexchange.com/questions/tagged/machine-learning), as this isn't a programming specific question, and has nothing to do with keras directly – DJK Aug 23 '17 at 16:37

1 Answers1

0

For scaling, I usually use the preprocessing.scale function from Scikit-Learn. It gives good results. For a deeper explanation should I normalize/standardize/rescale

For LSTM layers is advised to use "RELU" activation function it helps with vanishing gradient problem check this answer: advantages of RELU. For your second layer (last) use linear activation since you doing regression, if you were doing classification then use "sigmoid".

Germán Alfaro
  • 600
  • 6
  • 18