ValueError: could not broadcast input array from shape (300,300,3) into shape (300,300)

Question

I have a simple Python code (a Keras tutorial for training). I tried to remove img = img.convert('L') to keep colors when loading images (all my images are RGB colored so data is not the issue), but I encountered this error:

training_images = np.array([i[0] for i in training_data]).reshape(-1, IMAGE_SIZE, IMAGE_SIZE, 3)
ValueError: could not broadcast input array from shape (300,300,3) into shape (300,300)

What's going wrong? How to fix it?

from keras.models import Sequential, load_model
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.layers.normalization import BatchNormalization
from PIL import Image
from random import shuffle, choice
import numpy as np
import os

IMAGE_SIZE = 300
IMAGE_DIRECTORY = './data/test_set'

def label_img(name):
  if name == 'cats': return np.array([1, 0])
  elif name == 'notcats' : return np.array([0, 1])

def load_data():
  train_data = []
  directories = next(os.walk(IMAGE_DIRECTORY))[1]

  for dirname in directories:
    file_names = next(os.walk(os.path.join(IMAGE_DIRECTORY, dirname)))[2]

    for i in range(200):
      image_name = choice(file_names)
      image_path = os.path.join(IMAGE_DIRECTORY, dirname, image_name)
      label = label_img(dirname)
      img = Image.open(image_path)
      #img = img.convert('L')
      img = img.resize((IMAGE_SIZE, IMAGE_SIZE), Image.ANTIALIAS)
      train_data.append([np.array(img), label])

  return train_data

def create_model():
  model = Sequential()
  model.add(Conv2D(32, kernel_size = (3, 3), activation='relu', 
                   input_shape=(IMAGE_SIZE, IMAGE_SIZE, 1)))
  model.add(MaxPooling2D(pool_size=(2,2)))
  model.add(BatchNormalization())
  model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
  model.add(MaxPooling2D(pool_size=(2,2)))
  model.add(BatchNormalization())
  model.add(Dropout(0.2))
  model.add(Flatten())
  model.add(Dense(256, activation='relu'))
  model.add(Dropout(0.2))
  model.add(Dense(64, activation='relu'))
  model.add(Dense(2, activation = 'softmax'))

  return model

training_data = load_data()
training_images = np.array([i[0] for i in training_data]).reshape(-1, IMAGE_SIZE, IMAGE_SIZE, 1)
training_labels = np.array([i[1] for i in training_data])

model = create_model()
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(training_images, training_labels, batch_size=50, epochs=10, verbose=1)

Perhaps you can start with creating a [mre], as it stands there is a lot of noise in your shared code. Regardless, a shape of `(300, 300, 3)` vs `(300, 300)` contain vastly different number of elements (270000 vs 90000), that's why `numpy` is complaining it can't reshape the array. — r.ook, Nov 27 '19 at 19:26
OK. Shrinked even more. Wanted to share the context of training as well if that helps. — Tina J, Nov 27 '19 at 19:29
This question might be answered here: https://stackoverflow.com/questions/43977463/valueerror-could-not-broadcast-input-array-from-shape-224-224-3-into-shape-2 — aminrd, Nov 27 '19 at 20:04
Are you sure all images in your image directory are RGB (coloured) images or does it also contain greyscale images? In your exception it looks like `IMAGE_SIZE` is 300 instead of 256 (as in your code), is that correct? — Jonathan Feenstra, Nov 27 '19 at 20:33
Yeah sorry. I updated the 300 size. Yes all are RGB. I don't think that is the problem. I believe problem is `reshape(-1, IMAGE_SIZE, IMAGE_SIZE, 1)` — Tina J, Nov 27 '19 at 20:38
Idk. The reshape I thought was done for the grayscale version. Here is the tutorial: https://www.codeproject.com/Articles/4023566/Cat-or-Not-An-Image-Classifier-using-Python-and-Ke — Tina J, Nov 27 '19 at 20:52
Even if I pass one image, still get the error. It's not bc of the single image problem. — Tina J, Nov 27 '19 at 20:52
If your images are RGB, that should be `reshape(-1, IMAGE_SIZE, IMAGE_SIZE, 3)` instead of 1. 3-channel RGB images cannot be reshaped into single-channel images, otherwise you would lose the colour information, hence the error. In fact, the reshape should not be necessary at all, they should already be of that shape. — Jonathan Feenstra, Nov 27 '19 at 20:53
Yes I did the change of `reshape(-1, IMAGE_SIZE, IMAGE_SIZE, 3)` while changing it inside the model and removing the `convert(L)`. Still doesn't work. — Tina J, Nov 27 '19 at 21:33

score 2 · Accepted Answer · answered Nov 27 '19 at 22:21

Since I was able to identify the problem after some discussion in the comments, I will post it as an answer.

At the line

training_images = np.array([i[0] for i in training_data]).reshape(-1, IMAGE_SIZE, IMAGE_SIZE, 1)

you are attempting to reshape 3-channel RGB images into single channel (greyscale) images, which is not possible (and also not something you want to do, since you want to keep the colours), hence the ValueError. This part was only necessary before you removed img = img.convert('L'), in order to give the training data the proper shape for the model, which had an input shape of (IMAGE_SIZE, IMAGE_SIZE, 1).

Now that you are working with RGB images, the reshape can be removed, since the images will already have the correct shape (IMAGE_SIZE, IMAGE_SIZE, 3) as returned by load_data(). However, as explained in nneonneo's answer, your model will need to be modified to be able to handle the new input shape.

score 1 · Answer 2 · answered Nov 27 '19 at 20:40

1

  model.add(Conv2D(32, kernel_size = (3, 3), activation='relu', 
                   input_shape=(IMAGE_SIZE, IMAGE_SIZE, 1)))

Your model wants a grayscale image (1 channel), but you're trying to train on colour images (3 channels). This won't work. You will have to modify your model to take colour images, or pass in grayscale images. The sample code you started with uses .reshape(-1, IMAGE_SIZE, IMAGE_SIZE, 1) in order to convert a grayscale image into the shape required for the first layer of this neural net.

If the model is designed for grayscale, you should simply leave the .convert('L') in, which converts colour images to grayscale. Many image classification tasks work just fine in grayscale.

answered Nov 27 '19 at 20:40

nneonneo

171,345
36
312
383

I changed input_shape to 3 and it still didn't work. So what changes should I make? – Tina J Nov 27 '19 at 20:42
You'd have to change the reshape too. However, it's probably won't give you good results as the model is designed for grayscale. – nneonneo Nov 27 '19 at 20:43
Model accuracy is not issue right now (+ do you know what upgrades to make?). But right now it's how to pass the colored images without error. – Tina J Nov 27 '19 at 20:54
You should probably find a different model that takes coloured images. You could try simply changing `input_shape` and `reshape`, but that's not guaranteed to simply work - give it a shot? – nneonneo Nov 27 '19 at 20:57
I tried. Changed all the input_shapes to 3 and removed `.convert('L')`. But still errors. – Tina J Nov 27 '19 at 20:58
Btw, if you are a deep learnig guy, do you even think for fire detection I should use colored images? I guess it will help as it's all about orange color. – Tina J Nov 27 '19 at 21:06
1

@Tina J the error already occurs before fitting the model. While it is true that you need to modify your model if you want it to work with colour images, that is not the direct cause of the error. The cause is that you are trying to reshape a 3-channel image into a single-channel image, which is not possible (see my last comment on your question). Even if it was possible, you would lose the colour, because single-channel images are greyscale. – Jonathan Feenstra Nov 27 '19 at 21:09
@JonathanFeenstra Correct. So no way to change it? Assume the model is taking 3 channels as input. – Tina J Nov 27 '19 at 21:32
1

@Tina J if you remove the `reshape` part altogether (from the line `training_images = np.array([i[0] for i in training_data]).reshape(-1, IMAGE_SIZE, IMAGE_SIZE, 1)`), your array should contain 3-channel images. – Jonathan Feenstra Nov 27 '19 at 21:37
1

Heh yeah it's working! Thanks. But for grayscale version, I can't simply remove reshape. Otherwise I get `Input to `.fit()` should have rank 4. Got array with shape: (16, 300, 300)` error. – Tina J Nov 27 '19 at 21:42
1

@TinaJ yes, you would still need it for greyscale because the `input_shape` of the input layer is `(IMAGE_SIZE, IMAGE_SIZE, 1)` and not `(IMAGE_SIZE, IMAGE_SIZE)`. – Jonathan Feenstra Nov 27 '19 at 21:50
1

@JonathanFeenstra Thanks a lot. Also, if you have any suggestions for improving the model for colored version, let me know. I can ask it as a separate question also. – Tina J Nov 27 '19 at 21:53

ValueError: could not broadcast input array from shape (300,300,3) into shape (300,300)

2 Answers2