Why is AutoKeras max_model_size not working?

Question

I am trying to constrain the max params used by AutoKeras ImageClassifier to 1mil using max_model_size = 1000000. However, when I do this I immediately get the error: "FailedTrialError: Oversized model: 23538689 parameters. Skip model."

When I don't limit this I can see it does a first trial on a low param model, then goes straight to a resnet50 variant for trial 2 with 23mil params.

How do I fix this?

My code:

# Set max epochs and batch size
EPOCHS = 2 # keeping low while testing code
BATCH_SIZE = 128 # 256 caused OOM error on colab

# Metrics that will be reported
METRICS = [keras.metrics.AUC(name='prc', curve='PR'), # precision-recall curve]

# Early stopping
early_stopping = tf.keras.callbacks.EarlyStopping(
    monitor='val_prc', # metric it will monitor to stop training
    verbose=1,
    patience=10,
    mode='max',
    restore_best_weights=True)


# Initialize image classifier
clf = autokeras.ImageClassifier(
    overwrite=True,
    directory=pickle_file_dir,
    max_trials=2, # keeping low while testing code
    max_model_size = 100000000,
    metrics=METRICS,
    objective = keras_tuner.Objective("val_prc", direction="max"),
    project_name="bomb_finder1"
)

# Train the classifier
clf.fit(train_features, train_labels, 
        epochs=EPOCHS, 
        batch_size=BATCH_SIZE, 
        callbacks=[early_stopping], 
        #validation_split=0.15, # trains for extra run at end if using this, see: https://github.com/keras-team/autokeras/discussions/1344
        validation_data = (val_features, val_labels))

EDIT:

I'm now trying autokereas's AutoModel() function instead of the ImageClassifier(). If I try to set max_model_size = 500000 I get this error:

Best val_prc So Far: None
Total elapsed time: 00h 00m 01s
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-21-1bd6a59fdf09> in <module>
     28 
     29 # train the model
---> 30 clf.fit(train_features, train_labels, 
     31         epochs=EPOCHS,
     32         batch_size=BATCH_SIZE,

1 frames
/usr/local/lib/python3.8/dist-packages/autokeras/auto_model.py in fit(self, x, y, batch_size, epochs, callbacks, validation_split, validation_data, verbose, **kwargs)
    290             )
    291 
--> 292         history = self.tuner.search(
    293             x=dataset,
    294             epochs=epochs,

/usr/local/lib/python3.8/dist-packages/autokeras/engine/tuner.py in search(self, epochs, callbacks, validation_split, verbose, **fit_kwargs)
    221         else:
    222             # TODO: Add return history functionality in Keras Tuner
--> 223             model = self.get_best_models()[0]
    224             history = None
    225             pipeline = pipeline_module.load_pipeline(

IndexError: list index out of range

My code:

# define the model space

# input ...
image_input = autokeras.ImageInput()

# conv layers, chooses num layers, kernel size and count, whether to add max pooling and dropout
conv_layers = autokeras.ConvBlock()(image_input)

# dense layers ...
dense_layers = autokeras.DenseBlock(use_batchnorm=False)(conv_layers)

# output node ...
classification_head = autokeras.ClassificationHead(
    num_classes=1, multi_label=False, loss="binary_crossentropy", metrics=METRICS, dropout=None,
)(dense_layers)

# build the model
clf = autokeras.AutoModel(
    inputs=image_input, outputs=classification_head, 
    objective = keras_tuner.Objective("val_prc", direction="max"),
    tuner = 'hyperband',
    max_model_size = 500000,
    max_trials = 1,
    directory = pickle_file_dir,
    overwrite = True, 
    project_name="auto_model1"
)

# train the model
clf.fit(train_features, train_labels, 
        epochs=EPOCHS, 
        batch_size=BATCH_SIZE, 
        callbacks=[early_stopping], 
        validation_data = (val_features, val_labels))

Why is AutoKeras max_model_size not working?

0 Answers0