0

I'm tuning my models with GridSearchCV. There are multiple scoring metrics used - precision, recall and accuracy. However, when I inspect the cv_results_, all 3 metrics give the same score values.

Here is my code.

model = RandomForestClassifier(random_state=0)

param_grid = {
    'max_depth': range(5, 11, 1),
    'max_features': range(10, 21, 1),
    'n_estimators': [10, 100],
    'min_samples_leaf':[0.1, 0.2, 0.3]
}

scorers = {
    'precision_score': make_scorer(precision_score, average='micro'),
    'recall_score': make_scorer(recall_score, average='micro'),
    'accuracy_score': make_scorer(accuracy_score)
}

grid_search = GridSearchCV(model, param_grid = param_grid, 
                           scoring=scorers, refit='precision_score', return_train_score=True,
                          cv = 3, n_jobs = -1, verbose = 3)

grid_search.fit(X,y)

The scores returned by cv_results:

pd.DataFrame(grid_search.cv_results_)[['mean_test_precision_score', 'mean_test_recall_score', 'mean_test_accuracy_score']].head(10) enter image description here

The way I made my X and y (if that matters):

X = data_merge.drop('label', axis=1)
le = preprocessing.LabelEncoder()
y = le.fit_transform(data_merge['label'])

I'm running Scikit Learn 0.24.2 and Python 3.8.10

desertnaut
  • 57,590
  • 26
  • 140
  • 166
got2nosth
  • 578
  • 2
  • 8
  • 27
  • Wow, I've never seen this happen. Either there is really something spectacular about your data and model combination where precision, recall and accuracy actually come out to be the same (nearly impossible) or something in the way you are viewing the data? Could you also include the call to `cv_results`? – pu239 Aug 05 '21 at 01:50
  • @SamarthBhatia I didn't find anything spectacular about my data. I have tested it with different combinations of parameters in `GridSearchCV`, all yield the same results. The table I attached is the sample output from `cv_results` – got2nosth Aug 05 '21 at 03:02
  • 3
    https://stackoverflow.com/questions/62792001/precision-and-recall-are-the-same-within-a-model This explains it pretty well! – pu239 Aug 05 '21 at 03:34
  • @SamarthBhatia Thank you! You saved my day! Indeed setting `average` to `weighted` instead of `micro` gave me different precision and recall output. – got2nosth Aug 05 '21 at 04:00
  • you might also want to look at the [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_score.html#sklearn-metrics-precision-score), there are more options for `average` – pu239 Aug 05 '21 at 04:17
  • 2
    Another helpful answer in this context might be found [here](https://stackoverflow.com/questions/62469371/interpreting-auc-accuracy-and-f1-score-on-the-unbalanced-dataset/62471736#62471736) where I try to explain when to choose which averaging strategy. If you find this one and the answer provided above by @SamarthBhatia helpful, consider upvoting them. I see this issue coming up rather often, and it might make it easier for others to find this information. – afsharov Aug 05 '21 at 13:12
  • Thanks @afsharov. This is a very clear explanation of the different average strategies. I hope others could be benefited from it too. – got2nosth Aug 06 '21 at 12:19

0 Answers0