0

I'm doing the multiclass classification using Logistic Regression approach. Basically I know that if I use accuracy_score () function (for example, from sklearn library) it will calculate me the accuracy of distinct value to distinct value like this:

y_pred = [0, 2, 1, 3]
y_true = [0, 1, 2, 3]
accuracy_score(y_true, y_pred)
0.5

But I want to get the accuracy_score_new () function for a vector of top Logistic Regression predictions for each label (from predict_proba) and calculates whether the true label is in this interval like this:

y_pred = [[0,1,3] [2,1,4], [1,2,5] [3,7,9]]
y_true = [0, 1, 2, 3]
accuracy_score_new(y_true, y_pred)
1

The accuracy_score_new in this example will be equal to 1 because the classifier predicts that the label is in the interval. How can this function be done?

Keithx
  • 2,994
  • 15
  • 42
  • 71
  • Do you want to use the new function inside any scikit class or just want to use it yourself for printing the new accuracy. If latter, then you can write simple python code for it. Just iterate over both the arrays. – Vivek Kumar Aug 31 '17 at 12:20

2 Answers2

2

Accuracy is just (matching values /total values).

So in your case it will be something like:

def accuracy_score_new(y_pred, y_true):
    matched = 0
    for y_p, y_t in zip(y_pred, y_true):
        if y_t in y_p:
            matched = matched + 1

    return (matched / (float) len(y_true))
Vivek Kumar
  • 35,217
  • 8
  • 109
  • 132
1

Yes you can do that using the make_scorer function in sklearn. The idea is that you define your custom function assuming it gets the two parameters y_true and y_pred. You can also add any additional parameters if you want.

Here is an example : Custom scoring function

Here is another example : Using MSE and R2 score at the same time

This answer might be of some help too.

Gambit1614
  • 8,547
  • 1
  • 25
  • 51