How to plot a ROC curve from a softmax binary classifier with 2 output neurons?

Question

How to plot the roc curve with discrete outputs labels as 2 columns?

Using the roc_curve() gives me an error:

ValueError: multilabel-indicator format is not supported

y_prediction = model.predict(test_X)

y_prediction[1]
Out[27]: array([1.0000000e+00, 6.8178085e-12], dtype=float32)

y_prediction.shape
Out[23]: (514, 2)

test_y.shape
Out[24]: (514, 2)

fpr_roc, tpr_roc, thresholds_roc = roc_curve(test_y, y_prediction)

roc_auc = metrics.auc(fpr_roc, tpr_roc)

Answer 1

From the documentation, the y_true and y_score should be 1-d.

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html

y_truearray, shape = [n_samples]

So, just take the labels instead of softmax output.

Add the following lines before roc_curve()

test_y = np.argmax(test_y, axis=-1) # getting the labels
y_prediction = np.argmax(y_prediction, axis=-1) # getting the confidence of postive class

Answer 2

As roc_curve function in sklearn only requires positive class probability estimates, you can use the dimension of output, which is related to the positive class.

For example: preds[:, -1]