I'm trying to optimize my hyperparameters with Optuna, but I can't figure out how to tell Optuna thath the ENCODER NEURONS must be a list where length is not fixed (it's also an hyperparameter) and the values are integers (the first must be input_dim and the other are suggested). The code I tried is:
def objective(trial):
LATENT_DIM = trial.suggest_int("latent_dim",1,5)
HIDDEN_ACTIVATION = trial.suggest_categorical("hidden_activation", ["relu", "sigmoid", "tanh"])
EPOCHS = trial.suggest_int("epochs",1,100)
BATCH_SIZE = trial.suggest_int("batch_size",32,256,32)
DROPOUT_RATE = trial.suggest_float("dropout_rate", 0, 0.6)
L2_REGULARIZER = trial.suggest_float("l2_regularizer", 0, 0.6)
ENCODER_NEURONS = [input_dim] + [trial.suggest_int('encoder_neuron', 7, 256) for _ in range(trial.suggest_int('num_hidden_layers_encoder', 1, 5))]
DECODER_NEURONS = [trial.suggest_int('decoder_neuron', 7, 256) for _ in range(trial.suggest_int('num_hidden_layers_decoder', 1, 5))]+[input_dim]
model=VAE(encoder_neurons=ENCODER_NEURONS,decoder_neurons=DECODER_NEURONS,latent_dim=LATENT_DIM,
hidden_activation=HIDDEN_ACTIVATION,epochs=EPOCHS,batch_size=BATCH_SIZE,
dropout_rate=DROPOUT_RATE,l2_regularizer=L2_REGULARIZER,contamination=CONTAMINATION,random_state=RANDOM_STATE)
model.fit(x_train_scaled)
y_pred=model.predict(x_test_scaled)
score=fbeta_score(y_test,y_pred,beta=2)
return score
but it doesn't work
I would like to have as output a list like [7,16,32] or [7,32] or [7,16,32,64] where 7 is input_dim. But (part of) the output is 'num_hidden_layers_encoder': 2, 'encoder_neuron': 175, 'num_hidden_layers_decoder': 3, 'decoder_neuron': 99 that is not what I want. Someone can help me? Thanks
