2

I am mimicking the code from this page, implenting LSTM to predict time series behaviour. I use R so I just translated the content and adapted it to my dataset using R keras package.

So, here it goes, I have

train_input is a vector of random numbers between 8 and 21, let's say [9,10,19,17,...]. train_output is the shifted train_input vector by a one step : [10,19,17,...]. 

model <- keras_model_sequential()
model %>% layer_lstm(model,4,input_shape=c(1,1)%>% layer_dense(1)
model %>%compile(loss="mean_squared_error",optimizer="adam")
model %>% fit(train_input,train_output)
model %>% predict(test)

Here's I formatted the train_input in the expected 3D format ,by doing 

train_input <- array(train_input,dim=c(length(train_input),1,1))

Now, when I run mod%>% predict(test) for a test vector I get incoherent numbers between 0 and 1, as if probabilities have been computed. Do someone have an explanation ?

user2478159
  • 133
  • 1
  • 14
  • you should have an additional dense layer as the output for the network, not an `LSTM`, and try using activation function `'linear'` – DJK Aug 23 '17 at 15:43
  • @djk47463 sorry I got a typo when writing the post, indeed I have this `layer_dense(1)`. Is this what you meant ? – user2478159 Aug 23 '17 at 17:13
  • yes can you just add it – DJK Aug 23 '17 at 18:50
  • @djk47463 that's what I did but it changes nothing – user2478159 Aug 23 '17 at 19:29
  • Did you normalize the input data? – DJK Aug 23 '17 at 19:33
  • @djk47463 no I didn't, does it impact so much the result ? Besides, if you normalize the data before the training and then reverse the scaling, I understand that the linear part carries without problem the scale factor, but there are some non-linearity, how does this factor is dealt with ? – user2478159 Aug 23 '17 at 19:57
  • It can, think about the input gates activation of lstm is a sigmoid, so the activation is on a 0-1 scale. So in your sample a 19 will always carry more weight on the output of the network then say a 1, in the blog you posted he does scaling too. Normalizing is pretty standard for ANNs. – DJK Aug 23 '17 at 20:36

0 Answers0