I am using keras and tensorflow in R to fit a CNN composed of multi-input layers. I have an image dataset composed of photos of the height and width of an object and I want to apply a regression model. I am creating the training sets using flow_images_from_dataframe and compiling the models. However, I can't fit the model due to an error related to the input list.
train_generator <- keras::image_data_generator()
train_images.height = keras::flow_images_from_dataframe(
dataframe=train_df.height,
generator=train_generator,
y_col='volume',
class_mode="other",
target_size=c(253, 497),
batch_size = 10)
train_images.width = keras::flow_images_from_dataframe(
dataframe=train_df.width,
generator=train_generator,
y_col='volume',
class_mode="other",
target_size=c(253, 497),
batch_size = 10)
After this step, I am defining my CNN layers for both inputs and concatenating the layers:
input.height <- layer_input(shape = c(253, 497, 1), name="input_height")
conv.height<- input.height %>% layer_conv_2d(filters = 16, kernel_size = c(3, 3), activation = 'relu' ) %>%
layer_max_pooling_2d(pool_size = c(2, 2)) %>%
layer_conv_2d(filters = 32, kernel_size = c(3, 3), activation = 'relu') %>%
layer_max_pooling_2d(pool_size = c(2, 2)) %>%
layer_conv_2d(filters = 64, kernel_size = c(3, 3), activation = 'relu') %>%
layer_max_pooling_2d(pool_size = c(2, 2)) %>%
layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = 'relu') %>%
layer_max_pooling_2d(pool_size = c(2, 2))
input.width <- layer_input(shape = c(253, 497, 1), name="input_width")
conv.width<- input.width %>% layer_conv_2d(filters = 16, kernel_size = c(3, 3), activation = 'relu' ) %>%
layer_max_pooling_2d(pool_size = c(2, 2)) %>%
layer_conv_2d(filters = 32, kernel_size = c(3, 3), activation = 'relu') %>%
layer_max_pooling_2d(pool_size = c(2, 2)) %>%
layer_conv_2d(filters = 64, kernel_size = c(3, 3), activation = 'relu') %>%
layer_max_pooling_2d(pool_size = c(2, 2)) %>%
layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = 'relu') %>%
layer_max_pooling_2d(pool_size = c(2, 2))
combined<-keras::layer_concatenate(list(conv.height, conv.width))
combined.out <- combined %>%
layer_flatten() %>%
layer_dense(units = 512, activation = 'relu') %>%
layer_dense(units = 1,activation = 'linear')
combined <- keras_model(list(input.height, input.width), combined.out)
combined %>%
compile(optimizer='adam',
loss='mse')
However, when I try to fit the model using keras::fit() I got an error related to the input layers.
history <- combined %>% keras::fit(
list(train_images.height, train_images.width),
steps_per_epoch=nrow(rbind(train_df.height, train_df.width)) / 32,
epochs = 100,
callbacks = list(monitor='val_loss',
patience=5,
restore_best_weights=T))
Error in dim(x) <- length(x) :
invalid first argument, must be vector (list or atomic)
Is there anything that I should do with the train_images.height and train_images.width (generated using flow_images_from_dataframe) before I feed the fit function with both datasets as a list? I was using a similar code to fit a single input model and it was working well. The real model also includes the validation datasets, which were not included in the question to reduce the length.
