4

I have a keras model and I use sklearn.model_selection.GridSearchCV for tuning the hyperparameters, but it gets stuck in an infinite loop.

This is my model:

from keras import Sequential
from keras.layers import Dense, Activation

def make_model(optimizer='rmsprop'):
  model = Sequential()
  model.add(Dense(9, activation='relu', input_dim=28 * 28))
  model.add(Dense(27 , activation='relu'))
  model.add(Dense(27 , activation='relu'))
  model.add(Dense(81 , activation='relu'))
  model.add(Dense(10, activation='softmax'))
  model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics='acc')
  return model

from sklearn.model_selection import GridSearchCV
from keras.wrappers.scikit_learn import KerasClassifier

model = KerasClassifier(build_fn=make_model)

param_grid=dict(
    optimizer=['rmsprop'], 
    epochs=[25, 40], 
    batch_size=[45],
    validation_split=[.2])

grid_model = GridSearchCV(estimator=model, param_grid=param_grid)

And when I call fit on the model, instead of running with 25 and 40 epochs it will get stuck in an infinite loop.

I used keras.datasets.fashion_mnist as my dataset as below:

from keras.utils.np_utils import to_categorical


(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

y_train = to_categorical(train_labels, num_classes=10)
y_test = to_categorical(test_labels, num_classes=10)

x_train = train_images / 255.0
x_test  = test_images / 255.0

x_train = x_train.reshape(60000, -1)
x_test = x_test.reshape(10000, -1)

I have used epochs=[2, 3] to representing the loop, and rest of the code as the same as before.

It's the result: enter image description here

Moj Taba
  • 85
  • 6
  • Might this [Hyperparameter Tuning (Keras) a Neural Network Regression](https://stackoverflow.com/questions/70449758/hyperparameter-tuning-keras-a-neural-network-regression) help? – rickhg12hs Dec 28 '21 at 02:25
  • 1
    Could you please add a sample of your data and the first few lines of the console output where you see the infinite loop? The code included in the question is correct and the infinite loop issue is not reproducible with randomly generated data, I see the model being trained for 25 and 40 epochs as expected. – Flavia Giammarino Dec 28 '21 at 13:26
  • That's the expected output, it's not an infinite loop :). For instance, if you set the number of epochs equal to 3, then the model will go through the training set 3 times (that's why you see 1/3, 2/3, 3/3 printed in the console). Maybe take a look at the answers to [this question](https://stackoverflow.com/q/40069524/11989081). – Flavia Giammarino Dec 28 '21 at 19:09
  • As you see in the picture above the model was trained 3 times and it keeps going for 5 times just for `epochs=2`. But it should train the model for 1/2, 2/2 for `epochs=2` . Am I wrong? – Moj Taba Dec 28 '21 at 21:09
  • 1
    The model is trained and evaluated 5 times for each parameter combination, where 5 is the default number of cross-validation splits. You can change the number of splits when you define `GridSearchCV`. – Flavia Giammarino Dec 29 '21 at 06:10
  • Thanks, it worked, but It needs `cv >= 2`. Is there any way to train the model with `cv=1`? – Moj Taba Dec 29 '21 at 08:02
  • You can use [`train_test_split`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) to split the data into a training set and a test set, then fit the model to the training set and evaluate it on the test set. – Flavia Giammarino Dec 29 '21 at 16:00

0 Answers0