9

I am trying to optimize an SVR model using GridSearchCV and facing a problem because of overfitting, to overcome this I have tried to decrease the number of iterations instead of leaving it until convergence.To compare the both models I need the number of iterations for both cases.

I tried to do this through verbose = 1 but it didn't work in jupyter notebook. I need to access the number of iterations and save it as a variable to plot the results of optimization. Writing the verbose log to an external file could solve the problem but I am not able to do it.

I got some information from a previous question Knowing the number of iterations needed for convergence in SVR scikit-learn but still not able to solve the problem

Here is a sample of my code:

model_1=SVR(kernel='rbf',C=316,epsilon=0, gamma=0.003162,max_iter=2500,verbose=1)
model_1.fit(tr_sets[:,:2],tr_sets[:,2])
Community
  • 1
  • 1
Ahmad Sultan
  • 1,621
  • 2
  • 13
  • 19

2 Answers2

3

If I understand the logging in scikit learn correctly, they are not using the python logging infrastructure at all, but use print() directly, which makes capturing it extremely hard, especially in conjunction with jupyter notebooks, which do a lot to capture the standard streams.

Aha found it, it uses joblibs verbose printing infrastructure internally, which has the problem I mentioned.

https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/cross_validation.py

I have used this trick in the past to make joblib using the normall logging facility instead: https://github.com/joblib/joblib/issues/190 But I have no idea how to make that kind of monkey patching work in scikit learn

Christian Sauer
  • 10,351
  • 10
  • 53
  • 85
2

Taking into consideration what Christian stated in his answer, if logging by scikit-learn is done by the print(), to have it saved into an external file, you should redirect sys.stdout to a file of your wish and then any print() output will be printed to the file instead of the console.

To redirect sys.stdout to a file do the following:

import sys

sys.stdout=open("external_file.txt","w")
print ("this is a log message")

sys.stdout.close()

Now if you open external_file.txt, you will see the "this is a log message" text in the file

Don't forget to close the file at the end of the script! (sys.stdout.close())

bluesummers
  • 11,365
  • 8
  • 72
  • 108
  • 1
    Well this is the problem, the logging isn't done by a `print()` command. But thank you any way :) – Ahmad Sultan Jan 08 '17 at 17:55
  • 2
    Did anyone figure out how ti make this work? Changing `stdout` doesn't capture `GridSearchCV`'s logging to file. – ITA Jun 10 '18 at 18:45
  • 1
    According to this [answer](https://stackoverflow.com/a/34697183/1181141), sklearn uses joblib's "logging", which will send messages to `sys.stdout` if the verbosity level is above 50. So you can set `sys.stdout` to an open file, and use `verbose=51` in `GridSearchCV`. – mbil Jan 31 '19 at 02:24