ValueError: Error when checking target: expected avg_pool to have 4 dimensions, but got array with shape (100, 2)

Question

I try to run this code (binary classification) but still stuck with this error : ValueError: Error when checking target: expected avg_pool to have 4 dimensions, but got array with shape (100, 2)

    NUM_CLASSES = 2
    CHANNELS = 3
    IMAGE_RESIZE = 224
    RESNET50_POOLING_AVERAGE = 'avg'
    DENSE_LAYER_ACTIVATION = 'softmax'
    OBJECTIVE_FUNCTION = 'binary_crossentropy'
    NUM_EPOCHS = 10
    EARLY_STOP_PATIENCE = 3
    STEPS_PER_EPOCH_VALIDATION = 10
    BATCH_SIZE_TRAINING = 100
    BATCH_SIZE_VALIDATION = 100
    resnet_weights_path = '../input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5'
    train_data_dir = "C:\\Users\\Desktop\\RESNET"
    model = ResNet50(include_top=True, weights='imagenet')
    x = model.get_layer('avg_pool').output
    predictions = Dense(1, activation='sigmoid')(x)
    model = Model(input = model.input, output = predictions)
    print(model.summary())
    model.layers.pop() 
    model = Model(input=model.input,output=model.layers[-1].output)
    sgd = optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
    model.compile(loss='binary_crossentropy', optimizer=SGD(lr=0.01, momentum=0.9), metrics=   ['binary_accuracy'])
    data_dir = "C:\\Users\\Desktop\\RESNET"
    batch_size = 32
    from keras.applications.resnet50 import preprocess_input
    from keras.preprocessing.image import ImageDataGenerator
    image_size = IMAGE_RESIZE
    data_generator = ImageDataGenerator(preprocessing_function=preprocess_input)
    def append_ext(fn):
        return fn+".jpg"
    dir_path = os.path.dirname(os.path.realpath(__file__))
    train_dir_path = dir_path + '\data'
    onlyfiles = [f for f in listdir(dir_path) if isfile(join(dir_path, f))]
    NUM_CLASSES = 2
    data_labels = [0, 1]
    t = []
    maxi = 25145
    LieOffset = 15799
    i = 0
    while i < maxi: # t = tuple
    if i <= LieOffset:
        t.append(label['Lie'])
    else:
        t.append(label['Truth'])
    i = i+1
    train_datagenerator = ImageDataGenerator(rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True,
        validation_split=0.2) 
    train_generator = train_datagenerator.flow_from_directory(
        train_data_dir,
        target_size=(image_size, image_size), 
        batch_size=BATCH_SIZE_TRAINING,
        class_mode='categorical', shuffle=False, subset='training') # set as training data

    validation_generator = train_datagenerator.flow_from_directory(
        train_data_dir, # same directory as training data 
        target_size=(image_size, image_size), 
        batch_size=BATCH_SIZE_TRAINING,
        class_mode='categorical', shuffle=False, subset='validation') 

    (BATCH_SIZE_TRAINING, len(train_generator), BATCH_SIZE_VALIDATION, len(validation_generator))
    from sklearn.grid_search import ParameterGrid
    param_grid = {'epochs': [5, 10, 15], 'steps_per_epoch' : [10, 20, 50]}
    grid = ParameterGrid(param_grid)
# Accumulate history of all permutations (may be for viewing trend) and keep watching for lowest   val_loss as final model
    for params in grid:
    fit_history = model.fit_generator(
        train_generator,
        steps_per_epoch=STEPS_PER_EPOCH_TRAINING,
        epochs = NUM_EPOCHS,
        validation_data=validation_generator,
        validation_steps=STEPS_PER_EPOCH_VALIDATION,
        callbacks=[cb_checkpointer, cb_early_stopper]
    )
    model.load_weights("../working/best.hdf5")

Answer 1

Remove these lines and it'll work,

model.layers.pop() 
model = Model(input=model.input,output=model.layers[-1].output)

The former is removing the last(Dense) layer, and the letter means create a model without the last(Flatten, as Dense is already popped) layer.

It doesn't make sense as your target data is (100, 2). Why do you put them there in the first place?

Also I think this line

predictions = Dense(1, activation='sigmoid')(x)

Will error, as your target data is 2 channel, if it does error then change this to

predictions = Dense(2, activation='sigmoid')(x)

Update

The output of avg_pool is 4 dimention, (batch_size, height, width, channel). You need to do the Flatten first or use GlobalAveragePooling2D instead of AveragePooling2D.

Like

x = model.get_layer('avg_pool').output
x = keras.layers.Flatten()(x)
predictions = Dense(1, activation='sigmoid')(x)

Or

model = ResNet50(include_top=False, pooling='avg', weights='imagenet')  # `pooling='avg'` makes the `ResNet50` include a `GlobalAveragePoiling` layer and `include_top=False` means that you don't include the imagenet's output layer
x = model.output  # as I use `include_top=False`, you don't need to care the layer name, just use the model's output right away
predictions = Dense(1, activation='sigmoid')(x)

Also just as @bit01 said, change class_mode='categorical' to class_mode='binary'.

Answer 2

Change class_mode='categorical' to class_mode='binary' in both train_generator and validation_generator.

Additionally, delete the following lines as you have already created model.

model.layers.pop() 
model = Model(input=model.input,output=model.layers[-1].output)

So, your model would be like:

model = ResNet50(include_top=True, weights='imagenet')
x = model.get_layer('avg_pool').output
x = Flatten()(x)
predictions = Dense(1, activation='sigmoid')(x)
model = Model(input = model.input, output = predictions)