I am writing an example using Ray Tune with BayesOptSearch algorithm. However, I got an error at line "results = tuner.fit()" presents
BayesOpt does not support parameters of type
Categorical
I am running Ray 2.5.1 on Ubuntu. The problem may be the use of discrete parameters in Bayesian algorithms as blow:
{"param": tune.choice([20, 30])}
The complete code is as follows:
from ray import tune
from ray.air import session
from ray.tune.search.bayesopt import BayesOptSearch
def train_fn(config):
session.report({"loss": config["param"]})
tuner = tune.Tuner(
train_fn,
tune_config=tune.TuneConfig(
num_samples=100,
metric="loss",
mode="min",
search_alg=BayesOptSearch(),
),
param_space={"param": tune.choice([20, 30])},
)
results = tuner.fit()
I got an answer at
However, I don't know how to use it in Ray Tune.
------------EDIT------------------
When I only switch BayesOptSearch to OptunaSearch which uses default bayesian optimization, the result is OK. The code as follow:
from ray import tune
from ray.air import session
from ray.tune.search.bayesopt import BayesOptSearch
from ray.tune.search.optuna import OptunaSearch
def train_fn(config):
session.report({"loss": config["param"]})
tuner = tune.Tuner(
train_fn,
tune_config=tune.TuneConfig(
num_samples=10,
metric="loss",
mode="min",
# OptunaSearch which uses default bayesian optimization, OK
search_alg=OptunaSearch(),
# BayesOptSearch optimization, ERROR
# search_alg=BayesOptSearch(),
),
param_space={"param": tune.choice([20, 30])},
)
results = tuner.fit()
best_result = results.get_best_result("loss", "min", "last")
print(f"Best result config: {best_result.config}")