10

I have a multiclass classification task. When I run my script based on the scikit example as the follows:

classifier = OneVsRestClassifier(GradientBoostingClassifier(n_estimators=70, max_depth=3, learning_rate=.02))

y_pred = classifier.fit(X_train, y_train).predict(X_test)
cnf_matrix = confusion_matrix(y_test, y_pred)

I get this error:

File "C:\ProgramData\Anaconda2\lib\site-packages\sklearn\metrics\classification.py", line 242, in confusion_matrix
    raise ValueError("%s is not supported" % y_type)
ValueError: multilabel-indicator is not supported

I tried to pass the labels=classifier.classes_ to confusion_matrix(), but it doesn't help.

y_test and y_pred are as the follow:

y_test =
array([[0, 0, 0, 1, 0, 0],
   [0, 0, 0, 0, 1, 0],
   [0, 1, 0, 0, 0, 0],
   ..., 
   [0, 0, 0, 0, 0, 1],
   [0, 0, 0, 1, 0, 0],
   [0, 0, 0, 0, 1, 0]])


y_pred = 
array([[0, 0, 0, 0, 0, 0],
   [0, 0, 0, 0, 0, 0],
   [0, 0, 0, 0, 0, 0],
   ..., 
   [0, 0, 0, 0, 0, 1],
   [0, 0, 0, 0, 0, 1],
   [0, 0, 0, 0, 0, 0]])
YNR
  • 867
  • 2
  • 13
  • 28
  • Why do you have `y_pred` and `y_test` as one-hot encoded arrays? What are your original class labels? You should give your code, starting from how you did the transformation of your `y`. – Vivek Kumar Apr 28 '17 at 01:42
  • @VivekKumar I binarized `y_train` and `y_test` as `y_test = label_binarize(y_test, classes=[0, 1, 2, 3, 4, 5])` for `OneVsRestClassifier()`. – YNR Apr 28 '17 at 08:30
  • You should put your original classes (not binarized) into the `confusion_matrix`. You need to reverse transform your `y_pred` to get the original classes from it. – Vivek Kumar Apr 28 '17 at 09:34
  • @VivekKumar Thank you. I used non-binarized version and it solved. – YNR Apr 28 '17 at 09:57

3 Answers3

10

This worked for me:

y_test_non_category = [ np.argmax(t) for t in y_test ]
y_predict_non_category = [ np.argmax(t) for t in y_predict ]

from sklearn.metrics import confusion_matrix
conf_mat = confusion_matrix(y_test_non_category, y_predict_non_category)

where y_test and y_predict are categorical variables like one-hot vectors.

Azhar Khan
  • 3,829
  • 11
  • 26
  • 32
8

First you need to create the label output array. Lets say you have 3 classes: 'cat', 'dog', 'house' indexed: 0,1,2 . And the prediction for 2 samples is: 'dog', 'house'. Your output will be: 

y_pred = [[0, 1, 0],[0, 0, 1]]

run y_pred.argmax(1) to get: [1,2] This array stands for the original label indexes, meaning: ['dog', 'house']

num_classes = 3

# from lable to categorial
y_prediction = np.array([1,2]) 
y_categorial = np_utils.to_categorical(y_prediction, num_classes)

# from categorial to lable indexing
y_pred = y_categorial.argmax(1)
Naomi Fridman
  • 2,095
  • 2
  • 25
  • 36
0

I just subtracted the outputs y_test matrix from the prediction y_pred matrix while keeping the categorical format. In case of -1, I assumed a false negative while in case of 1, a false positive.

Next: 

if output_matrix[i,j] == 1 and predictions_matrix[i,j] == 1:  
    produced_matrix[i,j] = 2

Ending up with the following notation:

  • -1: false negative
  •  1: false positive
  •  0: true negative
  •  2: true positive

Finally, be performing some naive counting you can produce any confusion metric.

mcchran
  • 62
  • 3