I have a multiclass classification problem. Now, I want to get the predictions from my fit classifier (OneVsRestClassifier(XGBoost)
) to obtain the Area under the ROC curves in Scikit-Learn.
I noticed that my AUCs from roc_curve
and auc
are better when I use the array of prediction probabilities from .pred_proba()
than when I use the 'hard' predictions from .pred()
.
y_pred = predictor.predict_proba(X_test_outer_loop)
roc_auc = dict()
for i in range(len(classes): #Number of Classes for Multiclass Classification: 6
fpr[i], tpr[i], _ = roc_curve(y_test.iloc[:, i], y_pred[:, i])
roc_auc[i] = auc(fpr[i], tpr[i]) #AUC for every class
0: 0.5774011299435028, 1: 0.8211974110032363, 2: 0.6990099009900991, 3: 0.48424731182795705,, 4: 0.574652606681778, 5: 0.40591397849462363
y_pred = predictor.predict(X_test_outer_loop)
roc_auc = dict()
for i in range(len(classes): #Number of Classes for Multiclass Classification: 6
fpr[i], tpr[i], _ = roc_curve(y_test.iloc[:, i], y_pred[:, i])
roc_auc[i] = auc(fpr[i], tpr[i]) #AUC for every class
0: 0.49049707602339176, 1: 0.49238578680203043, 2: 0.6261595547309833, 3: 0.4701334816462736, 4: 0.49542775858565336, 5: 0.46632124352331605
How can that be explained? I figured that the .pred_proba()
operator also ends up selecting the class with the highest prediction probability, similar to .pred()
for the generation of the ROC.
I understand that classification thresholds can be adjusted for binary classification so that but I do not see how that would be possible for a OneVsRestClassifier
with six estimators.