0

I am new to Keras and have been using KerasTuner for the hyperparameters. This works very well, but I have not yet managed to tune the batch size. There is an official way from Keras, which was discussed here:

How to tune the number of epochs and batch_size?

I have tried to implement the code in this way:

class ANN: 

 def build_ann(self, hp):

        model = Sequential()

        for i in range(hp.Int('num_layers', 1, 10)): 
            model.add(LSTM(units=hp.Int('units_' + str(i), min_value=2, max_value=20, step=1), return_sequences=(i < hp.Int('num_layers', 1, 10) - 1)))
            model.add(Dropout(rate=hp.Float('dropout_' + str(i), 0, 0.5, step=0.1)))    
        model.add(Dense(units=hp.Int('units_last', min_value=2, max_value=20, step=1)))
        model.add(Dense(units=len(self.targets), activation='sigmoid'))

        model.compile(optimizer=Adam(hp.Choice('learning_rate', values=[1e-2, 1e-3, 1e-4])),
                    loss='mean_squared_error', metrics=['accuracy'])

        model.build(input_shape=(1, self.maxlen, len(self.features)))

        return model
    
    def fit(self, hp, model, *args, **kwargs):
        return model.fit(
            *args,
            batch_size=hp.Int('batch_size', 1, 10, step=16),
            **kwargs,
        )
    

tuner = keras_tuner.Hyperband(
    ann.build_ann,
    objective='val_accuracy',
    max_epochs=50,
    factor=2,
    overwrite=True,
    directory='my_dir2',
    project_name='my_project')

tuner.search(trainx, trainy, epochs=50, validation_split=0.2 )

Unfortunately, I don't think the batch size is varied. Does this function only work with tuner = RandomSearch()? Where does def fit come into play and how do I configure it correctly?

Thank very much!

AndyEverything
  • 23
  • 4

1 Answers1

0

The batch size isn't varied because you set the step size to 16 which exceeds your maximum (10). Try using this

batch_size=hp.Int('batch_size', 1, 10)

kkpnd
  • 1
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jun 01 '23 at 07:16