38

multilabel-indicator is not supported is the error message I get, when trying to run:

confusion_matrix(y_test, predictions)

y_test is a DataFrame which is of shape:

Horse | Dog | Cat
1       0     0
0       1     0
0       1     0
...     ...   ...

predictions is a numpy array:

[[1, 0, 0],
 [0, 1, 0],
 [0, 1, 0]]

I've searched a bit for the error message, but haven't really found something I could apply. Any hints?

cs95
  • 379,657
  • 97
  • 704
  • 746
Khaine775
  • 2,715
  • 8
  • 22
  • 51
  • 3
    Just wanted to add my two cents for anyone who's looking for the right way to visualize errors of multilabel classifiers: Your prediction array looks like from a **multiclass** classifier. A confusion matrix wouldn't be suitable for **multilabel** classification where multiple labels are predicted at once. – Thilo Sep 27 '18 at 13:07

4 Answers4

76

No, your input to confusion_matrix must be a list of predictions, not OHEs (one hot encodings). Call argmax on your y_test and y_pred, and you should get what you expect.

confusion_matrix(
    y_test.values.argmax(axis=1), predictions.argmax(axis=1))

array([[1, 0],
       [0, 2]])
cs95
  • 379,657
  • 97
  • 704
  • 746
14

The confusion matrix takes a vector of labels (not the one-hot encoding). You should run

confusion_matrix(y_test.values.argmax(axis=1), predictions.argmax(axis=1))
Joshua Howard
  • 876
  • 1
  • 12
  • 25
5

If you have numpy.ndarray you can try the following


import seaborn as sns

T5_lables = ['4TCM','WCM','WSCCM','IWCM','CCM']    

ax= plt.subplot()

cm = confusion_matrix(np.asarray(Y_Test).argmax(axis=1), np.asarray(Y_Pred).argmax(axis=1))
sns.heatmap(cm, annot=True, fmt='g', ax=ax);  #annot=True to annotate cells, ftm='g' to disable scientific notation

# labels, title and ticks
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels'); 
ax.set_title('Confusion Matrix'); 
ax.xaxis.set_ticklabels(T5_lables); ax.yaxis.set_ticklabels(T5_lables);

image

ِAbdalrahman M. Amer
  • 79
  • 1
  • 3
4
from sklearn.metrics import confusion_matrix

predictions_one_hot = model.predict(test_data)
cm = confusion_matrix(labels_one_hot.argmax(axis=1), predictions_one_hot.argmax(axis=1))
print(cm)

Output would be something like this:

[[298   2  47  15  77   3  49]
 [ 14  31   2   0   5   1   2]
 [ 64   5 262  22  94  38  43]
 [ 16   1  20 779  15  14  34]
 [ 49   0  71  33 316   7 118]
 [ 14   0  42  23   5 323   9]
 [ 20   1  27  32  97  13 436]]
Scott
  • 4,974
  • 6
  • 35
  • 62