Issue with KerasClassifier

Question

I'm building my deep learning model using the following code:

model = KerasClassifier(build_fn=create_model, verbose=0)

# neurons = [16, 64, 128, 256]
neurons = [16]
# batch_size = [10, 20, 50, 100]
batch_size = [10]
epochs = [10]
# activation = ['relu', 'tanh', 'sigmoid', 'hard_sigmoid', 'linear', 'exponential']
activation = ['sigmoid', 'relu']
# optimizer = ['SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adam', 'Adamax', 'Nadam']
optimizer = ['SGD', 'Adadelta', 'Adam']
loss = ['squared_hinge']

param_grid = dict(neurons=neurons, batch_size=batch_size, epochs=epochs, activation=activation, optimizer=optimizer, loss=loss)

I then want to use grid search to find the optimal classifier, so I run:

from sklearn.model_selection import GridSearchCV

grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1)
grid_result = grid.fit(X_train.astype('float32'), y_train.astype('float32'))

print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))
means = grid_result.cv_results_['mean_test_score']
stds = grid_result.cv_results_['std_test_score']
params = grid_result.cv_results_['params']
for mean, stdev, param in zip(means, stds, params):
    print("%f (%f) with: %r" % (mean, stdev, param))

What is the issue here? I keep getting an error stating:

ValueError: Invalid parameter activation for estimator KerasClassifier.
This issue can likely be resolved by setting this parameter in the KerasClassifier constructor:
`KerasClassifier(activation=sigmoid)`
Check the list of available parameters with `estimator.get_params().keys()`

try adding `activation=`sigmoind` to `GridSearchCV` and let us know what happens, Also, try adding this if it did not solve `lambda_parameter=0.01` — Sadra, Mar 07 '22 at 02:34
Both of these did not seem to do the trick, any idea if there is anything else I can do? — Oteirp, Mar 07 '22 at 02:44
Yeah that didn't seem to work, I'm not sure if the error has to do with switching to scikeras — Oteirp, Mar 07 '22 at 03:04
[This](https://stackoverflow.com/questions/70250928/activation-parameter-not-working-in-gridsearch) should answer your question. — Ro.oT, Aug 15 '23 at 14:22

score 0 · Answer 1 · answered Aug 15 '23 at 13:23

I had the same error yesterday, If you are using scikeras.wrappers.KerasClassifier, you will have to define your parameters like model__activation=['relu','softmax'] , model with 2 underscores then parameter name. see scikeras official documentation. https://adriangb.com/scikeras/stable/quickstart.html#grid-search

params={
'batch_size':[20,25],
'epochs':[50,70],
'model__neurons_1':[6,7],
'model__neurons_2':[4,3],
'model__activation':['relu','softmax'],
'model__optimizer':['adam','rmsprop'],
'model__dropout':[0.1,0.2]
}
def create_model(neurons_1,neurons_2,activation,optimizer,dropout):
    nn = tf.keras.Sequential()
    nn.add(tf.keras.layers.Input(shape=11))       
    nn.add(tf.keras.layers.Dense(units=neurons_1,
           activation=activation,kernel_initializer='glorot_uniform'))
    nn.add(tf.keras.layers.Dropout(rate=dropout))
    nn.add(tf.keras.layers.Dense(units=neurons_2,activation=activation))    
    nn.add(tf.keras.layers.Dropout(rate=dropout))
    nn.add(tf.keras.layers.Dense(units=1,activation='sigmoid'))
    nn.compile(optimizer=optimizer,loss='binary_crossentropy',
               metrics=['accuracy']) 
    return nn

model = KerasClassifier(model=create_model)

gs= GridSearchCV(estimator=model,param_grid=params,scoring='accuracy',cv=10,
                 n_jobs=-1,return_train_score=True,verbose=0)

Issue with KerasClassifier

1 Answers1