I am trying to use StratifiedKFold to do cross validation on my CNN that outputs multiple binary classifications. However, StratifiedKFold are unable to process multi label indicators.
skf = StratifiedKFold(n_splits=10, shuffle=True, random_state=42)
fold_scores = []
confusion_matrices = []
for fold, (train_index, test_index) in enumerate(skf.split(X, Y)):
print(f'Fold {fold+1}:')
X_train, X_test = X[train_index], X[test_index]
Y_train, Y_test = Y[train_index], Y[test_index]
# Build the model
model = create_model()
# Train the model
model.fit(X_train, Y_train, epochs=10, batch_size=32)
# Evaluate the model on the test data
y_pred = model.predict_classes(X_test)
y_pred_binary = (y_pred > 0.5).astype(int) # convert probabilities to binary predictions
confusion_matrices.append([confusion_matrix(Y_test[i:], y_pred_binary[i:]) for i in range(4)])
# Convert Y_test back to multilabel-indicator format for evaluation
scores = model.evaluate(X_test, Y_test, verbose=0)
fold_scores.append(scores)
Here is the error :
ValueError: Supported target types are: ('binary', 'multiclass'). Got 'multilabel-indicator' instead.
Is there any alternatives aside than StratifiedKFold to do cross validation on model with multiple binary classification with imbalance datasets? I use Keras and Tensorflow for my CNN.