Have you tried your own roc_auc scoring rule? It seems like you are passing labels instead of probabilities (you originally need) for roc_auc.
problem described in here:
Different result roc_auc_score and plot_roc_curve
Solutions for own scorers:
Grid-Search finding Parameters for AUC
Update2
Sorry, saw today that my introduction text from the notebook was missing lol
When calculating roc_auc_score you have the option (it doesnt matter, if it is with or without gridsearch, with or without pipeline) that you can pass it labels like (0/1) or probabilities like (0.995, 0.6655). The first should be easy available if you just convert your probas to labels. However that would result in a (straight reversed L) output plot. That looks sometimes ugly. the other option is to use predicted probabilites to pass them them to the roc_auc_score. That would result in a (staircase reversed L) output plot, which looks much better.
So what you first should test is, can you get a roc auc score with labels, with and without grid, if that is the case. You should then try to get probabilities. And there, I believe, you have to write your own scoring method, as the roc-auc_score in grid only serves labels, that would result in high roc_auc scores. I wrote something for you, so you can see the label approach:
import xgboost as xgb
from sklearn.metrics import confusion_matrix
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import roc_auc_score
cancer = load_breast_cancer()
X = cancer.data
y = cancer.target
xgb_model = xgb.XGBClassifier(objective="binary:logistic",
eval_metric="auc",
use_label_encoder=False,
colsample_bytree = 0.3,
learning_rate = 0.1,
max_depth = 5,
gamma = 10,
n_estimators = 10,
verbosity=None)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
xgb_model.fit(X_train, y_train)
preds = xgb_model.predict(X_test)
print(confusion_matrix(preds, y_test))
print ('ROC AUC Score',roc_auc_score(y_test,preds))
Gives:
[[51 2]
[ 3 87]]
ROC AUC Score 0.9609862671660424
Here you can see it is ridicoulous high.
If you wanna do it with grid:
get rid of this:
# Fit
_ = grid_cv.fit(X, y)
just grid_cv.fit(x, y)
fit is a method applied to grid_cv and results are stored within grid_cv
print(grid_cv.best_score_)
should deliver the auc as you already have defined it.
See also: different roc_auc with XGBoost gridsearch scoring='roc_auc' and roc_auc_score?
But this should also be ridicoulos high, as you will be probably serving labels instead of probas.
beware also of:
What is the difference between cross_val_score with scoring='roc_auc' and roc_auc_score?
And nobody hinders you to apply the roc-auc_score function to your grid_results...