I am trying to implement a Multi-layer Perceptron through the Keras package (and tensorflow) to run a fast MLP. I want to use Bayesian Optimization to train the algorithm's hyperparameters. I get an error message though, saying "ValueError: rate is neither scalar nor scalar tensor" and then it prints the random value for the dropout parameter from keras. I then also get an error from caret that "There were missing values in resampled performance measures." I can get the process to work for non caret/keras algorithms.
Here is my code applied to the iris dataset that should reproduce the error:
library(caret)
library(rBayesianOptimization) # to create cv folds and for bayesian optimisation
library(mlrMBO) # for bayesian optimisation
library(tensorflow)
library(keras)
iris$Speciesset=as.factor(iris$Species=="setosa")
levels(iris$Speciesset) = c("nonset","set")
rounds=5
#tunning via bays search
ctrl = trainControl(method = "cv", number = rounds,
summaryFunction = twoClassSummary,
classProbs = TRUE, search= "grid",
verboseIter=FALSE,savePredictions = "all")
# objective function: we want to maximise the log likelihood by tuning MLP parameters
obj.fun <- smoof::makeSingleObjectiveFunction(
name = "mlp_cv_bayes",
fn = function(x){
train_model = caret::train(Speciesset~.,
data=iris, trControl=ctrl,
metric="ROC", method="mlpKerasDropout",
tuneGrid= expand.grid(
size = x["size"],
dropout = x["dropout"],
batch_size = x["batch_size"],
lr = x["lr"],
rho = x["rho"],
activation = x["activation"],
decay = x["decay"]
))
train_model$results$ROC
},
par.set = makeParamSet(
makeIntegerParam("size", lower= 10, upper = 500),
makeNumericParam("dropout", lower= 0.1, upper = .9),
makeIntegerParam("batch_size", lower= 2000, upper = 15000),
makeNumericParam("lr", lower= 0.01, upper = .9),
makeNumericParam("rho", lower= 0.01, upper = .9),
makeNumericParam("decay", lower= 0.00001, upper = .9),
makeDiscreteParam("activation", values = c("relu", "tanh", "sigmoid"))
),
minimize = FALSE
)
control = makeMBOControl()
control = setMBOControlTermination(control, iters = 20)
des = generateDesign(par.set = getParamSet(obj.funnb),
fun = lhs::randomLHS)
run = mbo(fun = obj.fun,
control = control,
show.info = TRUE, design = des)
