ValueError: Error when checking input: expected Input_input to have 4 dimensions, but got array with shape (1, 1, 2)

Question

I am trying to create a Flappy Bird AI with Convolutional Layers and Dense Layers, but at the "Train" step (Function fit()) I get the following error message:

dqn.fit(env, nb_steps=500000, visualize=False, verbose=2)

Training for 500000 steps ...
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-53-e21cf8798454> in <module>()
----> 1 dqn.fit(env, nb_steps=500000, visualize=False, verbose=2) #fit = training, training for 5 Mio, timesteps eig bei 5000000
      2 #value's which are important: episode reward, mean reward

7 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/training_utils_v1.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
    634                            ': expected ' + names[i] + ' to have ' +
    635                            str(len(shape)) + ' dimensions, but got array '
--> 636                            'with shape ' + str(data_shape))
    637         if not check_batch_axis:
    638           data_shape = data_shape[1:]

ValueError: Error when checking input: expected Input_input to have 4 dimensions, but got array with shape (1, 1, 2)

I have found an example on the internet where only Dense Layers were used (Copyright (c) 2020 Gabriel Nogueira (Talendar)). I would like to build a network with Conv2D and Dense Layers, but something doesn't seem to fit.

The code is built as follows:

import sys
import os

import flappy_bird_gym 
env = flappy_bird_gym.make("FlappyBird-v0") #greyscale format 

env.action_space #Discrete(2)
env.observation_space #Box(-inf, inf, (2,), float32)

actions = env.action_space.n #2
obs = env.observation_space.shape[0] #2

#Network:
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D, Dropout, Input

import numpy as np
from tensorflow.keras.optimizers import Adam
import tensorflow as tf

#build model
def build_model(obs, actions):

  model = Sequential() 

  model.add(Conv2D(32, (8,8), name='Input', padding='same',input_shape=(1,obs,1)))
  model.add(MaxPooling2D((2,2), padding='same', name='maxpooling1'))

  model.add(Conv2D(64, (4,4), padding='same', activation='relu', name='Conv1'))
  model.add(MaxPooling2D((2,2), padding='same', name='maxpooling2'))
 
  model.add(Conv2D(64, (3,3), padding='same', activation='relu', name='Conv2'))
  model.add(MaxPooling2D((2,2), padding='same', name='maxpooling3'))
  
  model.add(Flatten())
  
  model.add(Dense(256, activation='relu', name='Dense1')) 
  model.add(Dense(actions, activation='linear',name='Output'))
  
  return model

model = build_model(obs, actions)
model.summary()

Model: "sequential_15"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 Input (Conv2D)              (None, 1, 2, 32)          2080      
                                                                 
 maxpooling1 (MaxPooling2D)  (None, 1, 1, 32)          0         
                                                                 
 Conv1 (Conv2D)              (None, 1, 1, 64)          32832     
                                                                 
 maxpooling2 (MaxPooling2D)  (None, 1, 1, 64)          0         
                                                                 
 Conv2 (Conv2D)              (None, 1, 1, 64)          36928     
                                                                 
 maxpooling3 (MaxPooling2D)  (None, 1, 1, 64)          0         
                                                                 
 flatten_20 (Flatten)        (None, 64)                0         
                                                                 
 Dense1 (Dense)              (None, 256)               16640     
                                                                 
 Output (Dense)              (None, 2)                 514       
                                                                 
=================================================================
Total params: 88,994
Trainable params: 88,994
Non-trainable params: 0
_________________________________________________________________

#RL
from rl.agents import DQNAgent
from rl.memory import SequentialMemory
from rl.policy import LinearAnnealedPolicy, EpsGreedyQPolicy

#build agent:
def build_agent(): 
  policy = LinearAnnealedPolicy(EpsGreedyQPolicy(), attr='eps', value_max=0.5, value_min=.0001, value_test=.0, nb_steps=6000000)
  memory = SequentialMemory(limit=100000, window_length=1)
  dqn = DQNAgent(model=model, memory=memory, policy=policy, #RL Algorithm
                enable_dueling_network=True, dueling_type='avg', #technique you use 
                nb_actions=actions, nb_steps_warmup=5000)
  return dqn
  
dqn = build_agent() 

#train:
from tensorflow.keras.optimizers import Adam
dqn.compile(Adam(lr=0.00025)) 

dqn.fit(env, nb_steps=500000, visualize=False, verbose=2) #here the error occurs

--> in the last line the error occurs

Does anyone know what I am doing wrong or what I need to change?

Answer 1

The error is coming from your input data.

As you can see the first layer is expecting the data to have dimension (None, 1, 2, 32) (The None is just the number of samples in the array). The key thing is your data has shape (1,2,2) and not (1, 2, 32). If you show us your data or maybe the what kind of data we can probably help a bit more on how to reshape it properly in order for the error to disappear.