1

I am trying to tune xgboost with hyperband and I would like to use the suggested default tuning space from the mlr3tuningspaces package. However, I don't find how to tag a hyperparameter with "budget" while using lts .

Below, I reproduced the mlr3hyperband package example to illustrate my issue:

library(mlr3verse)
library(mlr3hyperband)
library(mlr3tuningspaces)

## this does not work, because I don't know how to tag a hyperparameter 
## with "budget" while using the suggested tuning space
search_space = lts("classif.xgboost.default")
search_space$values

## this works because it has a hyperparameter (nrounds) tagged with "bugdget"
search_space = ps(
  nrounds = p_int(lower = 1, upper = 16, tags = "budget"), 
  eta = p_dbl(lower = 0, upper = 1),
  booster = p_fct(levels = c("gbtree", "gblinear", "dart"))
)

# hyperparameter tuning on the pima indians diabetes data set
instance = tune(
  method = "hyperband",
  task = tsk("pima"),
  learner = lrn("classif.xgboost", eval_metric = "logloss"),
  resampling = rsmp("cv", folds = 3),
  measures = msr("classif.ce"),
  search_space = search_space,
  term_evals = 100
)

# best performing hyperparameter configuration
instance$result
user2165907
  • 1,401
  • 3
  • 13
  • 28

1 Answers1

4

Thanks for pointing this out. I will add the budget tag to the default search space. Until then you can use this code.

library(mlr3hyperband)
library(mlr3tuningspaces)
library(mlr3learners)

# get learner with search space in one go
learner = lts(lrn("classif.xgboost"))

# overwrite nrounds with budget tag
learner$param_set$values$nrounds = to_tune(p_int(1000, 5000, tags = "budget"))

instance = tune(
  method = "hyperband",
  task = tsk("pima"),
  learner = learner,
  resampling = rsmp("cv", folds = 3),
  measures = msr("classif.ce"),
  term_evals = 100
)

Update 28.06.2022

The new API in version 0.3.0 is

learner = lts(lrn("classif.xgboost"), nrounds = to_tune(p_int(1000, 5000, tags = "budget"))
be-marc
  • 1,276
  • 5
  • 5
  • Thanks for the proposed workaround. When I run the line `learner = lts(lrn("classif.xgboost"))` it opens a debug window in RStudio, it works when I click "continue" but I thought I would mention it. – user2165907 Jun 24 '22 at 21:46
  • 1
    This bug is fixed in 0.3.0 – be-marc Jun 28 '22 at 11:11