I would like to use the geometric mean as a metric to optimize classification models trained with the fit() method. I thought that the autosklearn.metrics.make_scorer() would allow to define the geometric mean as a scorer like this:
classifier.fit(X_train, y_train, feat_type=feat_type, metric=autosklearn.metrics.make_scorer("gm", imblearn.metrics.geometric_mean_score))
as the imblearn-package is fully compatible with sklearn.
However, after the model has been "fitted" using the gm as defined above, the sprint statistics indicate that the definition as I did it does not seem to be working:
auto-sklearn results:
Dataset name: 6b31930a65e59cca700a5844fbab91a0
Metric: gm
Best validation score: 0.000000
Number of target algorithm runs: 187
Number of successful target algorithm runs: 82
Number of crashed target algorithm runs: 74
Number of target algorithms that exceeded the time limit: 18
Number of target algorithms that exceeded the memory limit: 13
Does anyone know why Auto-Sklearn seemed to be working with the GM but resulted in a score of 0,0000? Did I do anything wrong? Is it somehow possible to define the geometric mean as a metric to optimize the model other than using the imblearn package? The metric is as far as I know not integrated within sklearn (yet)? Or does anyone know any other way how I could potentially use the geometric mean within Auto-Sklearn?
