0

When using partial_fit at Scikit SGDClassifier the number of iteration for the convergence of the cost functions equals 1, as stated in the description:

Perform one epoch of stochastic gradient descent on given samples.
Internally, this method uses max_iter = 1. Therefore, it is not guaranteed that a minimum of the cost function is reached after calling it once. Matters such as objective convergence and early stopping should be handled by the user.

How can I increase max_iter such that my cost function is optimized properly and not just with one iteration? Or related to the scikit- description, how can I handle “objective convergence” and “early stopping” to my classifier using partial_fit?

Luk-StackOverflow
  • 321
  • 1
  • 5
  • 10

2 Answers2

1

You can simply execute the partial_fit() command repeatedly with the same data, e.g. with the same batch. Here is my code fragment, where I just programmed a loop around the partial_fit() command:

for i_iter in np.arange(iter_per_batch):
       clf.partial_fit(X_batch, y_batch, classes=[0,1])

The variable iter_per_batch defines the number of iterations.

Michael Mior
  • 28,107
  • 9
  • 89
  • 113
Luk-StackOverflow
  • 321
  • 1
  • 5
  • 10
-1

You can simply use the fit() method instead of the partial_fit() method and increase the max_iter by providing an integer value for the number of iterations you would like to have for the SGDClassifier. The default here is 1000 iterations.

Have a look at the documentation with the max_iter parameter: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html

Kim Tang
  • 2,330
  • 2
  • 9
  • 34
  • I need partial_fit(), because I am using it as an online learner, which requires that after each step the weights are updated based on the actual weight values. I think, that when I use fit() these weights are not updated based on the actual weight values. I think they are initialized. – Luk-StackOverflow Sep 04 '20 at 08:08
  • 1
    Hm, maybe I misunderstood it. Why don't you just try to build a loop around the partial_fit method? I think that should work too. – Kim Tang Sep 04 '20 at 08:14