2

I have a model that accepts a sequence of images as an input (None, n_step, 128, 128) (instead of a single image) where n_step is a fixed number 10. And I am using categorical_crossentropy for classification of four class problem. But I have an error as shown below

ValueError: A target array with shape (1342, 10, 4) was passed for an output of shape (None, 1, 4) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.

I understand from the error, that it is expecting a single image at a time. Is there any way where I can use this loss for a sequence of images?

Output form the model would also be a set of 10 labels

Edit:

n_steps = 10
feature_count = 4
def create_model():

    trajectory_input = Input(shape=(n_steps, feature_count), name='trajectory_input')
    image_input  = Input(shape=(128, 128, n_steps), name='image_input')

    x_aware = (Conv2D(32, kernel_size=(3, 3), activation='relu'))(image_input)
    x_aware = (BatchNormalization())(x_aware)
    x_aware = (MaxPooling2D(pool_size=(2, 2)))(x_aware)
    x_aware = (Dropout(0.25))(x_aware)

    x_aware = (Conv2D(64, kernel_size=(3, 3), activation='relu'))(x_aware)
    x_aware = (BatchNormalization())(x_aware)
    x_aware = (MaxPooling2D(pool_size=(2, 2)))(x_aware)
    x_aware = (Dropout(0.25))(x_aware)

    x_aware = (Conv2D(64, kernel_size=(3, 3), activation='relu'))(x_aware)
    x_aware = (BatchNormalization())(x_aware)
    x_aware = (MaxPooling2D(pool_size=(2, 2)))(x_aware)
    x_aware = (Dropout(0.25))(x_aware)

    x_aware = (Dense(64, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x_aware)
    x_aware = (BatchNormalization())(x_aware)
    x_aware = (Dropout(0.25))(x_aware)
    x_aware = Reshape((1, 12544))(x_aware)

    x = (Dense(32, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(trajectory_input)
    x = Reshape((1, 32*n_steps))(x)

    x = concatenate([x, x_aware])
    x = (Dense(64, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)
    x = (Dense(32, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)

    x_reg = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)
    x_class = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)

    x_reg = Reshape((2, 4))(x_reg)

    output_regression = (Dense(2, name='main_output'))(x_reg)
    output_class = (Dense(4, name='classification_output', activation='softmax'))(x_class)

    adam = Adam(lr=learning_rate)
    model = Model(inputs=[trajectory_input, image_input], outputs=[output_regression, output_class])
    model.compile(optimizer=adam, loss={'main_output': 'mse','classification_output': 'categorical_crossentropy'}, metrics={'main_output': [euc_dist_1, euc_dist_2], 'classification_output': 'accuracy'})
    model.summary()
    return model

Inputs are the images and their related information for regression task and outputs are labels and the next predictable value.

iamkk
  • 135
  • 1
  • 16
  • How are you calling `categorical_crossentropy`? – doctorlove Feb 17 '21 at 10:51
  • I have two output (classification and regression), so for classification I am just using that in compile like this `model.compile(optimizer=adam, loss={'main_output': 'mse','classification_output': 'categorical_crossentropy'})` – iamkk Feb 17 '21 at 10:53
  • I still would like more info to try to reporduce, e.g. about the inputs to your model. This qu https://stackoverflow.com/questions/58832251/valueerror-this-loss-expects-targets-to-have-the-same-shape-as-the-output has more info in the question - and clues in the answer – doctorlove Feb 17 '21 at 11:00
  • I edited the question with complete info on model. It would be great if you can suggest some solution. Thanks! – iamkk Feb 17 '21 at 11:11
  • 1
    I was able to resolve the issue. Thanks! – iamkk Feb 17 '21 at 12:43
  • 1
    Feel free to answer your own question if it might help someone in the future – doctorlove Feb 17 '21 at 13:31

1 Answers1

0

Problem was with the input given to the Dense and Reshape layer of classification task. Since the input was in (10,128,128), number of units should be 10*num_class. So changing this

x_reg = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)
x_class = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)

x_reg = Reshape((2, 4))(x_reg)

output_regression = (Dense(2, name='main_output'))(x_reg)
output_class = (Dense(4, name='classification_output', activation='softmax'))(x_class)

to this resolved the issue

x_reg = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)
x_class = (Dense(40, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)

x_reg = Reshape((2, 4))(x_reg)
x_class = Reshape((10,4))(x_class)

output_regression = (Dense(2, name='main_output'))(x_reg)
output_class = (Dense(4, name='classification_output', activation='softmax'))(x_class)
iamkk
  • 135
  • 1
  • 16