AttributeError: 'tuple' object has no attribute '__array_interface__'

Question

I am trying to train a DQNAgent on Atari Breakout Game for my academic course project. I have trouble with the fit function throwing this error message AttributeError: 'tuple' object has no attribute 'array_interface'. Can any one let me know how I should proceed to resolve this issue. I have posted the code below along with the error. You can try running the code in jupyter notebook. Thank you.

env = gym.make("BreakoutDeterministic-v4")
nb_actions = env.action_space.n

IMG_SHAPE = (84, 84)
WINDOW_LENGTH = 4

class ImageProcessor(Processor):
    def process_observation(self, observation):
        img = Image.fromarray(observation)
        img = img.resize(IMG_SHAPE)
        img = img.convert("L")
        img = np.array(img)
        return img.astype('uint8')  # saves storage in experience memory
    
    def process_state_batch(self, batch):

        processed_batch = batch.astype('float32') / 255.
        return processed_batch

    def process_reward(self, reward):
        return np.clip(reward, -1., 1.)

input_shape = (WINDOW_LENGTH, IMG_SHAPE[0], IMG_SHAPE[1])

model = Sequential()
model.add(Permute((2, 3, 1), input_shape=input_shape))

model.add(Convolution2D(32, (8, 8), strides=(4, 4),kernel_initializer='he_normal'))
model.add(Activation('relu'))
model.add(Convolution2D(64, (4, 4), strides=(2, 2), kernel_initializer='he_normal'))
model.add(Activation('relu'))
model.add(Convolution2D(64, (3, 3), strides=(1, 1), kernel_initializer='he_normal'))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dense(nb_actions))
model.add(Activation('linear'))
print(model.summary())

memory = SequentialMemory(limit=1000000, window_length=WINDOW_LENGTH)
processor = ImageProcessor()
policy = LinearAnnealedPolicy(EpsGreedyQPolicy(), attr='eps', value_max=1., value_min=.1, value_test=.05, nb_steps=1000000)

pip install keras-rl2==1.0.4

dqn = DQNAgent(model=model, nb_actions=nb_actions, policy=policy, memory=memory,
               processor=processor, nb_steps_warmup=50000, gamma=.99, target_model_update=10000,
              train_interval=4, delta_clip=1)

dqn.compile(Adam(learning_rate=.00025), metrics=['mae'])

weights_filename = 'dqn_breakout_weights.h5f'
checkpoint_weights_filename = 'dqn_' + "BreakoutDeterministic-v4" + '_weights_{step}.h5f'
checkpoint_callback = ModelIntervalCheckpoint(checkpoint_weights_filename, interval=100000)

model.load_weights("weights/dqn_BreakoutDeterministic-v4_weights_900000.h5f")

policy = LinearAnnealedPolicy(EpsGreedyQPolicy(), attr='eps', value_max=0.3, value_min=.1, value_test=.05, nb_steps=100000)

dqn = DQNAgent(model=model, nb_actions=nb_actions, policy=policy, memory=memory,
               processor=processor, nb_steps_warmup=50000, gamma=.99, target_model_update=10000)
dqn.compile(Adam(learning_rate=.00025), metrics=['mae'])

dqn.fit(env, nb_steps=500000, callbacks=[checkpoint_callback], log_interval=10000, visualize=False)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[15], line 15
     12 dqn.compile(Adam(learning_rate=.00025), metrics=['mae'])
     14 # And train the model
---> 15 dqn.fit(env, nb_steps=500000, callbacks=[checkpoint_callback], log_interval=10000, visualize=False)

File ~/.local/lib/python3.10/site-packages/rl/core.py:134, in Agent.fit(self, env, nb_steps, action_repetition, callbacks, verbose, visualize, nb_max_start_steps, start_step_policy, log_interval, nb_max_episode_steps)
    132 observation = deepcopy(env.reset())
    133 if self.processor is not None:
--> 134     observation = self.processor.process_observation(observation)
    135 assert observation is not None
    137 # Perform random starts at beginning of episode and do not record them into the experience.
    138 # This slightly changes the start position between games.

Cell In[4], line 4, in ImageProcessor.process_observation(self, observation)
      2 def process_observation(self, observation):
      3     # First convert the numpy array to a PIL Image
----> 4     img = Image.fromarray(observation)
      5     # Then resize the image
      6     img = img.resize(IMG_SHAPE)

File /usr/lib/python3/dist-packages/PIL/Image.py:2803, in fromarray(obj, mode)
   2764 def fromarray(obj, mode=None):
   2765     """
   2766     Creates an image memory from an object exporting the array interface
   2767     (using the buffer protocol).
   (...)
   2801     .. versionadded:: 1.1.6
   2802     """
-> 2803     arr = obj.__array_interface__
   2804     shape = arr["shape"]
   2805     ndim = len(shape)

AttributeError: 'tuple' object has no attribute '__array_interface__'

Answer 1

So I might not have a solution (Edit: I have one now), but I do have and explanation:

env.reset() used to return only an array as the observation. But there was a discussion to change the reset function to return the observation AND info. Finally in version v0.25 they implement it so now it always returns a tuple, breaking a lot of code like rl.agents.DQNAgent.fit() where env.reset is still l expected to return only observation

I was facing the same issue so a temporal solution would be editing rl.core.Agent.fit() yourself.

If you're working with a python virtual env, go to lib\site-packages\rl\core.py, if you installed global in your system then go to AppData\Local\Programs\Python\PythonXXX\lib\site-packages... And in Agent.fit() use find and replace with every instance of observation = deepcopy(env.reset()) and replace it with observation, info = deepcopy(env.reset()) and it should work.

Answer 2

The error is suggesting that the observation object you're trying to convert to a PIL image is not a numpy array but a tuple. The fromarray function of the PIL Image module only takes numpy arrays as input.

The problem seems to be originating from the line where you call Image.fromarray(observation).

The observation variable comes from observation = self.processor.process_observation(observation) where observation is initially set to env.reset(). Now, usually in Gym environments, env.reset() returns a numpy array, which you should be able to convert to a PIL image.

One possible explanation for the error could be that the environment you are using, "BreakoutDeterministic-v4", is returning a tuple instead of a numpy array when reset() is called. This might be due to a custom wrapper or modification you've made to the environment.

Here are some steps to debug this issue:

Check the output of env.reset() directly: Try to print the output of env.reset() to see what it's actually returning. It should return a numpy array. If it's a tuple, try to figure out why that's the case.

Check the version of your Gym library: Make sure you are using a version of the Gym library that is known to work with your code. If you've recently updated Gym, it might be worth trying to roll back to a previous version.

Try a different environment: To check if the problem is with the environment or your code, try using a different environment and see if you get the same error.

Check your ImageProcessor class: The process_observation() function of the ImageProcessor class should return a numpy array. Make sure that it's not returning a tuple.

Check if you have any wrappers around your environment: If you've wrapped your environment with any custom Gym wrappers, make sure they're not modifying the output of reset() to be a tuple instead of a numpy array.