0

I want to evaluate my Keras model with confusion matrix. However, I can not make it work because I'm always getting the same error:

ValueError: Classification metrics can't handle a mix of multilabel-indicator and continuous-multioutput targets

I'm looking at this question:

confusion matrix error "Classification metrics can't handle a mix of multilabel-indicator and multiclass targets"

I have tried to mimic everything but it does not work. I think that this is not the same case.

This is my code:

validationTweets = validation.content.tolist() #data for validation
validation_features = vectorizerCV.transform(validationTweets) #vectorizing data for validation

prediction = model.predict(validation_features , batch_size=32) # making prediction

realLabels = validation.sentiment # true labels/classes (STRING VALUES)
realLabels = np.asarray(realLabels.factorize()[0]) # converting them to categorical
realLabels = to_categorical(realLabels, num_classes = 3) # converting them to categorical

print('true labels type', type(realLabels))  #<class 'numpy.ndarray'>
print('true labels shape',realLabels.shape)  # (5000, 3)

print('prediction type', type(prediction))  #<class 'numpy.ndarray'>
print('prediction shape', prediction.shape) #(5000, 3)


matrix = confusion_matrix(realLabels, prediction)

This is how my real labels look like:

[[1. 0. 0.]
 [1. 0. 0.]
 [0. 1. 0.]
 ...
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]

This is how my prediction looks like:

[[8.6341507e-04 6.8435425e-01 3.1478229e-01]
 [8.4774427e-02 7.8772342e-01 1.2750208e-01]
 [4.3412593e-01 5.0705791e-01 5.8816209e-02]
 ...
 [9.1305929e-01 6.6390157e-02 2.0550590e-02]
 [8.2271063e-01 1.5146920e-01 2.5820155e-02]
 [1.7649201e-01 7.2304797e-01 1.0045998e-01]]

I have tried this:

prediction = [np.round(p, 0) for p in prediction]

ERROR: multilabel-indicator is not supported

I have also tried this:

prediction = prediction.argmax(axis = 1) # shape is (5000,)

ERROR:    ValueError: Classification metrics can't handle a mix of multilabel-indicator and continuous-multioutput targets

But I'm getting the same error.

taga
  • 3,537
  • 13
  • 53
  • 119
  • Try with predictions = prediction.argmax(axis = 1), then pass predictions to cm. I assume they are softmax outputs. – Frightera Jan 29 '21 at 23:09
  • Does not work, I'm getting the same error, and the new shape of prediction is (5000,) – taga Jan 29 '21 at 23:13

1 Answers1

0

I'm not super familiar with Keras confusion_matrix, but rounding your predictions won't work for multiclass. For each sample, you need to take the highest probability predicted class, and make that entry equal to one in your prediction. For example:

pred = np.array([0.3, 0.3, 0.4])
rounded = np.round(pred) # Gives [0, 0, 0]
most_likely = np.zeros(3)
most_likely[pred >= np.max(pred)] = 1 # Gives [0,0,1]
converseg
  • 13
  • 3