Error "Negative dimension size caused by subtracting 3 from 2" in CNN for mnist dataset

Question

I'm designing a CNN for mnist dataset. Then it returns an error

ValueError: Negative dimension size caused by subtracting 3 from 2 for '{{node conv2d_23/Conv2D}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](batch_normalization_19/cond/Identity, conv2d_23/Conv2D/ReadVariableOp)' with input shapes: [?,2,2,12], [3,3,12,12].

I tried to change padding = 'valid' to padding = 'same', but the problem persists. Could you please elaborate on this error and how to fix it?

from keras.datasets import mnist
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Model
from keras.layers import Conv2D, Dense, Dropout, MaxPooling2D, Flatten, Input, Reshape, BatchNormalization
from keras.utils import to_categorical
import numpy as np
from keras.optimizers import SGD, Adam

X = Input(shape = (28, 28, 1))

for k in range(3):
    X = Conv2D(filters = 3 * (2 ** k), kernel_size = (3, 3), activation = 'relu', kernel_initializer = 'he_uniform')(X)
    X = BatchNormalization()(X)
    X = Conv2D(filters = 3 * (2 ** k), kernel_size = (3, 3), activation = 'relu', kernel_initializer = 'he_uniform')(X)
    X = BatchNormalization()(X)
    X = MaxPooling2D(pool_size = (2, 2), strides = (2, 2), padding = 'valid')(X)            
    X = Dropout(0.25 + (0.1 * k))(X)

X = Flatten()(X)
Y = Dense(units = 512, activation = 'relu', kernel_initializer = 'he_uniform')(X)
X = BatchNormalization()(X)
X = Dropout(0.5)(X)
Y = Dense(units = 10, activation = 'softmax')(X)

model = Model(X, Y)
model.compile(loss = 'binary_crossentropy',
              optimizer = 'Adam',
              metrics = ['accuracy'])

(x_train, y_train), (x_test, y_test) = mnist.load_data() 
# Reshape dataset to match the expected dimensions of 2D Convolution
x_train = x_train.reshape(x_train.shape[0], 28, 28, 1)
x_test = x_test.reshape(x_test.shape[0], 28, 28, 1)

y_train = to_categorical(y_train, n_classes)
y_test = to_categorical(y_test, n_classes)
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

history = model.fit(datagen.flow(x_train, y_train, batch_size = 32),
                    epochs = 10,
                    validation_data = (x_test, y_test))

plt.plot(history.history['loss'], 'tab:blue',
         history.history['val_loss'], 'tab:orange')
plt.legend(['loss', 'val_loss'])
loss_path = os.path.join(OUTDIR, lossplot_filename)
plt.savefig(loss_path)
plt.show()

Did you also try to set `same` padding for CNNs? Also might not be related to the problem but your loss function is not correct as you are doing multi-class classification. — Frightera, Mar 05 '21 at 15:38
@Frightera I change `padding = 'same'` and `loss = 'categorical_crossentropy'`, but the problem persists. — Akira, Mar 05 '21 at 15:42
@gobrewers14 I tried to add `input_shape=(28,28,1), data_format='channels_last'`, but the problem persists. — Akira, Mar 05 '21 at 15:54
You use max pooling with strides of 2, three times. After the first time, tensor shape is `28//4` so 7, after three times it's reduced to less than zero — Nicolas Gervais, Mar 05 '21 at 15:54
Thank you @NicolasGervais. I change the strides to 1 and it works :)) — Akira, Mar 05 '21 at 16:07

Error "Negative dimension size caused by subtracting 3 from 2" in CNN for mnist dataset

0 Answers0