5

So I have the following code-

params = {'n_estimators': [1000, 2000], 'max_depth': [10, 20], 'min_samples_split': [2, 3],
          'learning_rate': [0.1, 0.05, 0.01], 'loss': ('ls', 'huber', 'lad', 'quantile'), 'verbose': [1]}
gbr = ensemble.GradientBoostingRegressor()
clf = GridSearchCV(gbr, params)

And, although I don't think I've given it much to contemplate, it's taking FOREVER to determine the best parameters. After watching for a couple hours I've seen great instances (verbose:[1]) and wish to stop it from continuing. I don't want to have to come back to this in the morning and hope that it's finished. I want to finish it now without losing the hours of tuning in the process.

Is there a safe way I can stop the tuning and still have the best result?

Ravaal
  • 3,233
  • 6
  • 39
  • 66
  • 1
    I dont think so. The best you can hope is setting the verbose to maybe 3 and then searching the output for all the scores upto the point of interrupt and then finding the best yourself. – Vivek Kumar Mar 04 '18 at 03:13

2 Answers2

3

I use the following approach to get optimal hyperparameters for DNN (which you can use for yourself):

  • for loop for parameter #1
  • inside for loop for parameter #2
  • ...
  • inside you fit your model and save/print results, pictures etc.

Sure, if you ctrl-c your running code your pictures, logs will no be removed.

But as I see here, you do not have much possible parameter combinations inside. So why it runs forever:

  1. Data is extremely big (take subsample)
  2. Number of cv-s is very big (take less than 10)
  3. Reduce # of estimators. 1000-2000 is too much
  4. Take small max_features
  5. Reduce max_depth. 10-20 is too much I believe. For most cases it should be <10 (try with 3-5 for example)
  6. Use XGBoost (it's a lot faster). In general, XGBoost and GBR are the same (and both tend not to overfit with a growing number of estimators) but XGBoost has some additions that helps in this better.
  7. Run on some server or upgrade your PC :)
avchauzov
  • 1,007
  • 1
  • 8
  • 13
2

If this question is anything to go by, then the answer appears to be no. I know when I did this a few weeks back, I too got impatient. I just ctrl+f'ed through the logs for numbers I was hoping it would achieve. Starting at 95,94,93,... etc.

Tryer
  • 120
  • 1
  • 7