2

So I'm trying to practice how to use LSTMs in Keras and all parameter (samples, timesteps, features). 3D list is confusing me.

So I have some stock data and if the next item in the list is above the threshold of 5 which is +-2.50 it buys OR sells, if it is in the middle of that threshold it holds, these are my labels: my Y.

For my features my X I have a dataframe of [500, 1, 3] for my 500 samples and each timestep is 1 since each data is 1 hour increment and 3 for 3 features. But I get this error:

ValueError: Error when checking model input: expected lstm_1_input to have 3 dimensions, but got array with shape (500, 3)

How can I fix this code and what am I doing wrong?

import json
import pandas as pd
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM

"""
Sample of JSON file
{"time":"2017-01-02T01:56:14.000Z","usd":8.14},
{"time":"2017-01-02T02:56:14.000Z","usd":8.16},
{"time":"2017-01-02T03:56:15.000Z","usd":8.14},
{"time":"2017-01-02T04:56:16.000Z","usd":8.15}
"""
file = open("E.json", "r", encoding="utf8")
file = json.load(file)

"""
If the price jump of the next item is > or < +-2.50 the append 'Buy or 'Sell'
If its in the range of +- 2.50 then append 'Hold'
This si my classifier labels
"""
data = []
for row in range(len(file['data'])):
    row2 = row + 1
    if row2 == len(file['data']):
        break
    else:
        difference = file['data'][row]['usd'] - file['data'][row2]['usd']
        if difference > 2.50:
            data.append((file['data'][row]['usd'], 'SELL'))
        elif difference < -2.50:
            data.append((file['data'][row]['usd'], 'BUY'))
        else:
            data.append((file['data'][row]['usd'], 'HOLD'))

"""
add the price the time step which si 1 and the features which is 3
"""
frame = pd.DataFrame(data)
features = pd.DataFrame()
# train LSTM
for x in range(500):
    series = pd.Series(data=[500, 1, frame.iloc[x][0]])
    features = features.append(series, ignore_index=True)

labels = frame.iloc[16000:16500][1]

# test
#yt = frame.iloc[16500:16512][0]
#xt = pd.get_dummies(frame.iloc[16500:16512][1])


# create LSTM
model = Sequential()
model.add(LSTM(3, input_shape=features.shape, activation='relu', return_sequences=False))
model.add(Dense(2, activation='relu'))
model.add(Dense(1, activation='relu'))

model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])


model.fit(x=features.as_matrix(), y=labels.as_matrix())

"""
ERROR
Anaconda3\envs\Final\python.exe C:/Users/Def/PycharmProjects/Ether/Main.py
Using Theano backend.
Traceback (most recent call last):
  File "C:/Users/Def/PycharmProjects/Ether/Main.py", line 62, in <module>
    model.fit(x=features.as_matrix(), y=labels.as_matrix())
  File "\Anaconda3\envs\Final\lib\site-packages\keras\models.py", line 845, in fit
    initial_epoch=initial_epoch)
  File "\Anaconda3\envs\Final\lib\site-packages\keras\engine\training.py", line 1405, in fit
    batch_size=batch_size)
  File "\Anaconda3\envs\Final\lib\site-packages\keras\engine\training.py", line 1295, in _standardize_user_data
    exception_prefix='model input')
  File "\Anaconda3\envs\Final\lib\site-packages\keras\engine\training.py", line 121, in _standardize_input_data
    str(array.shape))
ValueError: Error when checking model input: expected lstm_1_input to have 3 dimensions, but got array with shape (500, 3)
"""

Thanks.

petezurich
  • 9,280
  • 9
  • 43
  • 57
Definity
  • 691
  • 2
  • 11
  • 31

1 Answers1

0

This is my first post here I wish that could be useful I will try to do my best

First you need to create 3 dimension array to work with input_shape in keras you can watch this in keras documentation or in a better way: from keras.models import Sequential Sequential? Linear stack of layers.

Arguments

layers: list of layers to add to the model.

# Note The first layer passed to a Sequential model should have a defined input shape. What that means is that it should have received an input_shape or batch_input_shape argument, or for some type of layers (recurrent, Dense...) an input_dim argument.

Example

```python
    model = Sequential()
    # first layer must have a defined input shape
    model.add(Dense(32, input_dim=500))
    # afterwards, Keras does automatic shape inference
    model.add(Dense(32))

    # also possible (equivalent to the above):
    model = Sequential()
    model.add(Dense(32, input_shape=(500,)))
    model.add(Dense(32))

    # also possible (equivalent to the above):
    model = Sequential()
    # here the batch dimension is None,
    # which means any batch size will be accepted by the model.
    model.add(Dense(32, batch_input_shape=(None, 500)))
    model.add(Dense(32))

After that how to transform arrays 2 dimensions in 3 dimmension check np.newaxis

Useful commands that help you more than you expect:

  • Sequential?, -Sequential??, -print(list(dir(Sequential)))

Best

mitillo
  • 1
  • 1
  • 1