2

I'm playing around with machine learning and trying to follow along with some examples but am stuck trying to get my data into a Keras LSTM layer.

I have some stock ticker data in a Pandas DataFrame which is resampled at 15 minute intervals with ohlc and a load of other metrics for each row.

My code is below. df is my DataFrame:

x = df.iloc[:, :-1].values
y = df.iloc[:, -1:].values

dimof_input = x.shape[1]
dimof_output = len(set(y.flat))

model = Sequential()
model.add(LSTM(4, input_dim=dimof_input, return_sequences=True))
model.compile(loss='mse', optimizer='rmsprop')

model.fit(x, y, nb_epoch=1, batch_size=1, verbose=2)

When I try and fit I get:

Error when checking input: expected lstm_16_input to have 3 dimensions, 
but got array with shape (33, 100)

I've copied this from examples elsewhere. I can't quite see how to get the correct shape of data into this model. Can anyone help?

Thanks loads.

Ludo
  • 2,739
  • 2
  • 28
  • 42
  • What are you actually trying to achieve with LSTM? – DarkCygnus Jul 07 '17 at 17:00
  • I'm following some tutorials for stock price forecasting. Mainly it's just as a problem to muck about with though rather than anything serious. – Ludo Jul 07 '17 at 17:02
  • Ok... maybe it has to do with your `dimof_input` which you get by doing `x.shape[1]` ... What is the shape of `x`? Usually I just do `x.shape` without the index... – DarkCygnus Jul 07 '17 at 17:04
  • @Ludo Did you manage to figure this out? Would like to see how to get from a df to 3D tensor... – cJc Mar 14 '18 at 14:56

1 Answers1

4

Input shapes

3D tensor with shape (batch_size, timesteps, input_dim), (Optional) 2D tensors with shape (batch_size, output_dim). (from there).
And you specified input_dim=dimof_input. Model expects 3D tensor as input, but got 2D. If you give a reference to the tutorial that is being implemented I probably will be able to say more about causes of the problem.
You could try reshape your input data as following:

x = x.reshape(x.shape[0], 1, x.shape[1])

Also, some information about input data of Keras LSTM layer input data could be found here.

kvorobiev
  • 5,012
  • 4
  • 29
  • 35
  • Thanks! Trying your suggested reshaping now changes the error to: ValueError: Error when checking input: expected lstm_17_input to have shape (None, None, 1) but got array with shape (33, 1, 100) I think I must be fundamentally misunderstanding how these input shapes work. I've now managed to lose the tutorial I was following along with. Let me try and find it. Thanks for your help. – Ludo Jul 10 '17 at 16:59