I'm working on a DQN model that trains on a CustomEnv from OpenAi Gymnasium. My observation space has just one dimension, with shape (8,) and that's going to be the input of my neural network. I first used a model with full dense layers like so:
def build_model(env):
model = Sequential()
input_shape = (1, env.observation_space.shape[0]) # this results in (1,8)
model.add(Flatten(input_shape=input_shape))
model.add(Dense(32, activation='relu'))
model.add(Dense(32, activation='relu'))
n_output_nodes = env.action_space.n
model.add(Dense(n_output_nodes, activation='linear'))
return model
As you can see I had to add and additional dimension to the network since I read that Keras adds an extra dimension for the batch as part of its processing. That's why my expected input shape is set like (1,8) instead of just (8,). This one worked well (sort of) and it didn't give me any error with the shape.
What I don't understand is how to set it with LSTM layers. The LSTM model is being declared as so:
def build_model_v2(env):
obs_shape = env.observation_space.shape
n_output_nodes = env.action_space.n
model = Sequential()
model.add(Reshape((1, obs_shape[0], 1), input_shape=(1, 1, obs_shape[0], 1))) # (1,1,8,1)
model.add(LSTM(32, return_sequences = True, activation = 'relu'))
model.add(Dense(24, activation = "relu"))
model.add(Flatten())
model.add(Dense(n_output_nodes, activation="linear"))
model.compile(loss="mse", optimizer=Adam(lr=0.0002), metrics=['accuracy'])
return model
My first layer is now a Reshape layer since I saw somebody with the same issue, who solved it by adding that layer. I declare now the input_shape on that layer. I read that LSTM's need and input shape of 3 dimensions for Samples, Time Steps and Features, meaning it would be an input_shape = (1,x,y,z), since I still have to add that extra dimension for keras processing, right?
But I'm getting this error:
ValueError: Input 0 of layer "lstm_20" is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: (None, 1, 8, 1)
What am I doing wrong? And can somebody explain to me what's that Samples, Time Steps and Features
