0

I am developing an algorithm based on gradient descent and I would like to add early stoping regularization. I have an objectif function,F, and I minimize it with respect to W. This is given in the code below:

Data : X_Train, Y_Train
t=1;
while (t < MaxIteration):
   W = W - step * Grad (F,X,W).
   loss(t) = computeLoss(X,Y,W);
end

Now I want to add early stoping regularization : this technique would consist in choosing the moment when it is necessary to stop during the optimization process (break the loop). How should I choose this moment? I have to test my model for each iteration on the validation data and create a history? What I'm trying to do is given below:

Data: X_Train, Y_Train, X_val, Y_val;
t=1;
maxIteration = 100;
models = array of size maxIteration
while (t < MaxIteration):
    W = W - step * Grad(F,X,W).
    loss(t) = computeLoss(X,Y,W);
    models(t) = W;
    t=t+1;
end

How do I choose the W model among all those I have stored?

ZchGarinch
  • 295
  • 3
  • 13

0 Answers0