I am using imblearn.keras.balanced_batch_generator
in my CNN classification task. But the generator produces inconsistent classes for my data (I have 12 classes in total but it produces 10/11/12 classes when batches are yielded).
This is causing an error when I train my model, indicating that my output layer node does not match with the class number (which matches this post: CNN model Categorical error: logits and labels must be broadcastable: logits_size=[32,10] labels_size=[32,13])
Code:
gen = BalancedBatchGenerator(data[["image"]], data[["class"]],
sampler=RandomOverSampler(),
random_state = 10,
batch_size=32)
for x,y in gen:
print(len(y.value_counts())) # check number of class, dataset is in dataframe format
# sample output
# 12
# 11
# 10
# 12
history = model.fit_generator(
generator = train_generator,
validation_data = val_generator,
epochs = 50,
verbose = 1,
callbacks = callbacks
)
Error:
InvalidArgumentError: logits and labels must be broadcastable: logits_size=[32,11] labels_size=[32,12]
[[node categorical_crossentropy/softmax_cross_entropy_with_logits (defined at <ipython-input-23-af8ee67b3eab>:6) ]] [Op:__inference_train_function_78283]