0

I am trying to use StratifiedKFold to do cross validation on my CNN that outputs multiple binary classifications. However, StratifiedKFold are unable to process multi label indicators.

skf = StratifiedKFold(n_splits=10, shuffle=True, random_state=42)

fold_scores = []
confusion_matrices = []

for fold, (train_index, test_index) in enumerate(skf.split(X, Y)):
    print(f'Fold {fold+1}:')
    X_train, X_test = X[train_index], X[test_index]
    Y_train, Y_test = Y[train_index], Y[test_index]

    # Build the model
    model = create_model()

    # Train the model
    model.fit(X_train, Y_train, epochs=10, batch_size=32)

    # Evaluate the model on the test data
    y_pred = model.predict_classes(X_test)
    y_pred_binary = (y_pred > 0.5).astype(int) # convert probabilities to binary predictions
    confusion_matrices.append([confusion_matrix(Y_test[i:], y_pred_binary[i:]) for i in range(4)])

    # Convert Y_test back to multilabel-indicator format for evaluation
    scores = model.evaluate(X_test, Y_test, verbose=0)
    fold_scores.append(scores)

Here is the error :

ValueError: Supported target types are: ('binary', 'multiclass'). Got 'multilabel-indicator' instead. 

Is there any alternatives aside than StratifiedKFold to do cross validation on model with multiple binary classification with imbalance datasets? I use Keras and Tensorflow for my CNN.

Albert
  • 1
  • 2

0 Answers0