6

I am trying to use OnClassSVM for anomaly detection purpose and I tuned its parameters using GridSearchCV() as follows:

I have searched many sites for it including https://stackoverflow.com/ but could not find any proper solution of my scenario. code is here: 

nus = [0.001, 0.01, 0.1, 1]
gammas = [0.001, 0.01, 0.1, 1]
scorers = {
  'precision_score': make_scorer(precision_score),
  'recall_score': make_scorer(recall_score),
 'accuracy_score': make_scorer(accuracy_score)
}

tuned_parameters = {'C': [1, 10, 100, 1000], 'kernel' : ['rbf','linear'], 
'gamma' : gammas, 'nu': nus}
tuned_ocsvm = svm.OneClassSVM()
ocsvm = GridSearchCV(estimator=svm.OneClassSVM(), 
param_grid=tuned_parameters, scoring=scorers,refit='false')

But it is giving me error as follows

For multi-metric scoring, the parameter refit must be set to a scorer key or a callable to refit an estimator with the best parameter setting on the whole data and make the best_* attributes available for that metric. If this is not needed, refit should be set to False explicitly. 'false' was passed

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Mani
  • 127
  • 1
  • 2
  • 7

2 Answers2

4

On GridSearchCV's doc, refit is defined as:

refit : boolean, string, or callable, default=True

Refit an estimator using the best found parameters on the whole dataset. For multiple metric evaluation, this needs to be a string denoting the scorer that would be used to find the best parameters for refitting the estimator at the end. Where there are considerations other than maximum score in choosing a best estimator, refit can be set to a function which returns the selected best_index_ given cv_results_. The refitted estimator is made available at the best_estimator_ attribute and permits using predict directly on this GridSearchCV instance. Also for multiple metric evaluation, the attributes best_index_, best_score_ and best_params_ will only be available if refit is set and all of them will be determined w.r.t this specific scorer. best_score_ is not returned if refit is callable. See scoring parameter to know more about multiple metric evaluation.

If you don't want to refit the estimator, you can set refit=False (as boolean). On the other hand, to refit the estimator with one of the scorer, you can do refit='precision_score' for example.

phoenix
  • 78
  • 1
  • 5
  • thankx phoenix for you replay I use refit='precision_score'. but now it is giving error as "missing 1 required positional argument: 'y_true'" – Mani Sep 18 '19 at 07:28
  • OneClassSVM is unsupervised and you don't have y_true. So you cannot evaluate the model in this way. What's the problem you're trying to solve? – phoenix Sep 18 '19 at 08:25
1

It's better to set your scorers to one of the scorer and do not pass it as a dictionary.

scorers = make_scorer(precision_score)

If you set refit=False then you will lose the best_estimator_ which is one of the most important outputs of grid search.

Morteza Mashayekhi
  • 934
  • 11
  • 23