1

I have a multiclass classification problem. Now, I want to get the predictions from my fit classifier (OneVsRestClassifier(XGBoost)) to obtain the Area under the ROC curves in Scikit-Learn. I noticed that my AUCs from roc_curve and auc are better when I use the array of prediction probabilities from .pred_proba() than when I use the 'hard' predictions from .pred().

 y_pred = predictor.predict_proba(X_test_outer_loop)
 roc_auc = dict()
    for i in range(len(classes):  #Number of Classes for Multiclass Classification: 6
                fpr[i], tpr[i], _ = roc_curve(y_test.iloc[:, i], y_pred[:, i])
                roc_auc[i] = auc(fpr[i], tpr[i]) #AUC for every class


  0: 0.5774011299435028, 1: 0.8211974110032363, 2: 0.6990099009900991, 3: 0.48424731182795705,, 4: 0.574652606681778, 5: 0.40591397849462363


 y_pred = predictor.predict(X_test_outer_loop)
     roc_auc = dict()
        for i in range(len(classes):  #Number of Classes for Multiclass Classification: 6
                    fpr[i], tpr[i], _ = roc_curve(y_test.iloc[:, i], y_pred[:, i])
                    roc_auc[i] = auc(fpr[i], tpr[i]) #AUC for every class

 0: 0.49049707602339176, 1: 0.49238578680203043, 2: 0.6261595547309833, 3: 0.4701334816462736, 4: 0.49542775858565336, 5: 0.46632124352331605

How can that be explained? I figured that the .pred_proba() operator also ends up selecting the class with the highest prediction probability, similar to .pred() for the generation of the ROC. I understand that classification thresholds can be adjusted for binary classification so that but I do not see how that would be possible for a OneVsRestClassifier with six estimators.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Leon J
  • 31
  • 3
  • 1
    Might https://stackoverflow.com/questions/70278059/plotting-the-roc-curve-for-a-multiclass-problem/70279497#70279497 help? – amiola Jun 16 '22 at 21:25

0 Answers0