How to get the result auc using scikit

Question

Hi i want to combine train/test split with a cross validation and get the results in auc.

My first approach I get it but with accuracy.

# split data into train+validation set and test set
X_trainval, X_test, y_trainval, y_test = train_test_split(dataset.data, dataset.target)
# split train+validation set into training and validation sets
X_train, X_valid, y_train, y_valid = train_test_split(X_trainval, y_trainval)
# train on classifier
clf.fit(X_train, y_train)
# evaluate the classifier on the test set
score = svm.score(X_valid, y_valid)
# combined training & validation set and evaluate it on the test set
clf.fit(X_trainval, y_trainval)
test_score = svm.score(X_test, y_test)

And I do not find how to apply roc_auc, please help.

How many classes do you have in your data? – ginge Jan 22 '17 at 17:28 — ginge, Jan 22 '17 at 17:28
Hi ginge, I have two classes 0-1 – xav Jan 23 '17 at 19:13 — xav, Jan 23 '17 at 19:13

score 0 · Answer 1 · answered Jan 23 '17 at 19:50

Using scikit-learn you can do:

import numpy as np
from sklearn import metrics
y = np.array([1, 1, 2, 2])
scores = np.array([0.1, 0.4, 0.35, 0.8])
fpr, tpr, thresholds = metrics.roc_curve(y, scores, pos_label=2)

Now we get:

print(fpr)

array([ 0. , 0.5, 0.5, 1. ])

print(tpr)

array([ 0.5, 0.5, 1. , 1. ])

print(thresholds)

array([ 0.8 , 0.4 , 0.35, 0.1 ])

score 0 · Answer 2 · answered May 17 '19 at 15:03

In your code, after training your classifier, get the predictions with:

y_preds = clf.predict(X_test)

And then use this to calculate the auc value:

from sklearn.metrics import roc_curve, auc

fpr, tpr, thresholds = roc_curve(y, y_preds, pos_label=1)
auc_roc = auc(fpr, tpr)

How to get the result auc using scikit

2 Answers2