0

I'm testing to tune parameters of SVM with hyperopt library. Often, when i execute this code, the progress bar stop and the code get stuck. I do not understand why.

Here is my code : 

from hyperopt import fmin, tpe, hp, STATUS_OK, Trials

X_train = normalize(X_train)

def hyperopt_train_test(params):

    if 'decision_function_shape' in params:
        if params['decision_function_shape'] == "ovo":
            params['break_ties'] = False


    clf = svm.SVC(**params)
    y_pred = clf.fit(X_train, y_train).predict(X_test)
    return precision_recall_fscore_support(y_test, y_pred, average='macro')[0]

space4svm = {
    'C': hp.uniform('C', 0, 20),
    'kernel': hp.choice('kernel', ['linear', 'sigmoid', 'poly', 'rbf']),
    'degree': hp.uniform('degree', 10, 30),
    'gamma': hp.uniform('gamma', 10, 30),
    'coef0': hp.uniform('coef0', 15, 30),
    'shrinking': hp.choice('shrinking', [True, False]),
    'probability': hp.choice('probability', [True, False]),
    'tol': hp.uniform('tol', 0, 3),
    'decision_function_shape': hp.choice('decision_function_shape', ['ovo', 'ovr']),
    'break_ties': hp.choice('break_ties', [True, False])
    }

def f(params):
    print(params)
    precision = hyperopt_train_test(params)
    return {'loss': -precision, 'status': STATUS_OK}

trials = Trials()
best = fmin(f, space4svm, algo=tpe.suggest, max_evals=35, trials=trials)
print('best:')
print(best)
Clement Ros
  • 329
  • 1
  • 3
  • 17

1 Answers1

1

I would suggest restricting the space of your parameters and see if that works. Fix the probability parameter to False and see if the model trains. Also, gamma needs to be {‘scale’, ‘auto’} according to the documentation.

Also at every iteration print out your params to better understand which combination is causing the model to get stuck.

Shaunak Sen
  • 548
  • 3
  • 8
  • Hi thanks for the answer, params coef seems to be the root cause. For gamma the doc says "{‘scale’, ‘auto’} or float," . But when i use hp.choice between {'scale', 'auto'} it's ok and when i use hp.uniform process get stuck. Secondly process get stuck every time i tried to test multiple params for coef0. My solution is use coef0 default value wich is 0.0 and use gamma scale or auto – Clement Ros Feb 07 '20 at 09:29
  • I have another question. How to choose max_evals value in fmin() ? Because this parameters can change the best loss value significatively – Clement Ros Feb 07 '20 at 09:32
  • Not entirely sure about this, but I think the problem occurs when `kernel='linear'` and we explicitly try and specify a value for `gamma`. Gamma is defined as Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid kernels. Coming to the question on choosing `max_evals`, it really depends on your problem and data. Maybe set a reasonable value (around 30) to get a reasonable set of parameters and then try again on a restricted subspace to get the optimal set of parameters.. – Shaunak Sen Feb 10 '20 at 15:25