2

I'd like to do a hyperparameter-tuning on a Keras model with Keras tuner.

import tensorflow as tf
from tensorflow import keras
import keras_tuner as kt

def model_builder(hp):

  model = keras.Sequential()
  model.add(keras.layers.Flatten(input_shape=(28, 28)))

  hp_units = hp.Int('units', min_value=32, max_value=512, step=32)
  model.add(keras.layers.Dense(units=hp_units, activation='relu'))
  model.add(keras.layers.Dense(10))

  hp_learning_rate = hp.Choice('learning_rate', values=[1e-2, 1e-3, 1e-4])

  model.compile(optimizer=keras.optimizers.Adam(learning_rate=hp_learning_rate),
                loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
                metrics=['accuracy'])

  return model

tuner = kt.Hyperband(model_builder,
                     objective='val_accuracy',
                     max_epochs=10,
                     factor=3)

tuner.search(train_X, train_y, epochs=50)

So far, so good. However, I additionally want to define some model parameters (like input image dimensions) as input parameters for model_builder, I'm clueless, how to done:

def model_builder(hp, img_dim1, img_dim2):

  model = keras.Sequential()
  model.add(keras.layers.Flatten(input_shape=(img_dim1, img_dim2)))
...

and

tuner = kt.Hyperband(model_builder(img_dim1, img_dim2),
                     objective='val_accuracy',
                     max_epochs=10,
                     factor=3)

seemingly doesn't work. How to feed img_dim1, img_dim2 to the model beyond hp?

Fredrik
  • 411
  • 1
  • 3
  • 14

2 Answers2

0

A simple solution is using "partial function" in python like the following:

from functools import partial

#...

model_builder_ready = partial(model_builder, img_dim1 = value1, img_dim2 = value2)

tuner = kt.Hyperband(model_builder_ready,
                     objective='val_accuracy',
                     max_epochs=10,
                     factor=3)
OmG
  • 18,337
  • 10
  • 57
  • 90
  • Weird: tuner = RandomSearch( model_builder_ready, objective=Objective("val_f1_m", direction="max"), max_trials=2, executions_per_trial=2 ) gives error: TypeError: model_builder() got multiple values for argument 'img_dim1' – Fredrik Nov 01 '21 at 06:16
  • @Fredrik Yeah, it's weird! You need to debug your code more carefully. – OmG Nov 01 '21 at 13:54
0

The solution I came up with was to create a function that return a function (probably what partial does), so this should look like this :

def model_builder(img_dim1, img_dim2):
    def func(hp):
        """
        Your original builder but here img_dim1 and img_dim2 exist in the scope so you can use them as parameter
        """
    return func

tuner = kt.Hyperband(model_builder(img_dim1, img_dim2),
                     objective='val_accuracy',
                     max_epochs=10,
                     factor=3)
rapido
  • 1